2024-09-16T14:03:37.782 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T14:03:37.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:03:37.887 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723 branch: reef description: rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906723' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - CEPHADM_AGENT_DOWN log-only-match: - CEPHADM_ sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdsaRAr0FKgcSOrtQy1XjGJfeMMPHJ05Sb9E+R3Fkw3xuAU5e4F7bKTHd1QljbRdAlSNgFCSrmdxsVslaBMyAY= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHrKgC4CQpQRn9I5XYE8bf4w+BjK06MPRZaqvAWBbsb300mmHfkkI+YXC8LykfJETucGmt3W+LKgtJIYX/e49fU= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T14:03:37.888 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T14:03:37.888 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T14:03:37.889 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T14:03:37.890 INFO:teuthology.task.internal:Checking packages... 2024-09-16T14:03:37.913 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T14:03:37.913 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T14:03:37.913 INFO:teuthology.packaging:ref: None 2024-09-16T14:03:37.913 INFO:teuthology.packaging:tag: None 2024-09-16T14:03:37.913 INFO:teuthology.packaging:branch: reef 2024-09-16T14:03:37.914 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:03:37.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T14:03:38.172 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853.gae7f3733 2024-09-16T14:03:38.173 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T14:03:38.181 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T14:03:38.181 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T14:03:38.219 INFO:teuthology.task.internal:Saving configuration 2024-09-16T14:03:38.229 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T14:03:38.243 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T14:03:38.267 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 13:58:31.236275', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdsaRAr0FKgcSOrtQy1XjGJfeMMPHJ05Sb9E+R3Fkw3xuAU5e4F7bKTHd1QljbRdAlSNgFCSrmdxsVslaBMyAY='} 2024-09-16T14:03:38.289 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 13:58:31.238276', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHrKgC4CQpQRn9I5XYE8bf4w+BjK06MPRZaqvAWBbsb300mmHfkkI+YXC8LykfJETucGmt3W+LKgtJIYX/e49fU='} 2024-09-16T14:03:38.289 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T14:03:38.295 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T14:03:38.295 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-16T14:03:38.295 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T14:03:38.374 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faa2edc7ac0>, signals=[15]) 2024-09-16T14:03:38.375 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T14:03:38.381 INFO:teuthology.task.internal:Opening connections... 2024-09-16T14:03:38.381 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T14:03:38.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:03:38.460 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-16T14:03:38.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:03:38.537 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T14:03:38.543 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-09-16T14:03:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-09-16T14:03:38.562 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:03:38.620 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:03:38.621 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:03:38.621 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:03:38.621 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:03:38.621 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:03:38.621 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-09-16T14:03:38.641 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-16T14:03:38.659 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-16T14:03:38.660 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:03:38.718 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:03:38.719 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:03:38.719 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:03:38.719 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-16T14:03:38.740 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T14:03:38.749 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T14:03:38.775 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T14:03:38.775 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:03:38.778 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:03:38.795 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T14:03:38.801 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T14:03:38.801 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:03:38.835 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:03:38.861 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T14:03:38.910 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T14:03:38.924 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T14:03:38.924 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T14:03:38.925 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:03:38.925 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:03:38.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:03:38.942 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-16T14:03:38.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:03:38.943 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-16T14:03:38.999 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:03:38.999 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.14.0-503.el9.x86_64 2024-09-16T14:03:38.999 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2024-09-16T14:03:39.001 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:03:39.001 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.14.0-503.el9.x86_64 2024-09-16T14:03:39.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-09-16T14:03:42.479 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-16T14:03:43.444 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-16T14:03:44.856 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-16T14:03:45.294 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-16T14:03:49.157 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-16T14:03:50.178 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.5 MB 00:00 2024-09-16T14:03:51.365 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:01 2024-09-16T14:03:51.928 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-16T14:03:52.636 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-16T14:03:53.574 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-16T14:03:58.067 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-09-16T14:03:59.163 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T14:04:00.065 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:04:00.065 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:04:00.170 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:04:00.175 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:04:00.176 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 79 M 2024-09-16T14:04:00.177 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 126 M 2024-09-16T14:04:00.177 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-16T14:04:00.518 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 77 kB/s | 23 kB 00:00 2024-09-16T14:04:01.277 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:04:01.277 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:04:01.365 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:04:01.376 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout:Install 4 Packages 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:04:01.377 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 79 M 2024-09-16T14:04:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 126 M 2024-09-16T14:04:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-09-16T14:04:01.497 INFO:teuthology.orchestra.run.smithi012.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 329 kB/s | 23 kB 00:00 2024-09-16T14:04:02.514 INFO:teuthology.orchestra.run.smithi012.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 28 MB/s | 29 MB 00:01 2024-09-16T14:04:02.831 INFO:teuthology.orchestra.run.smithi012.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 25 MB/s | 35 MB 00:01 2024-09-16T14:04:03.110 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-09-16T14:04:03.948 INFO:teuthology.orchestra.run.smithi012.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-09-16T14:04:03.948 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:04:03.948 INFO:teuthology.orchestra.run.smithi012.stdout:Total 31 MB/s | 79 MB 00:02 2024-09-16T14:04:04.652 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.9 MB/s | 29 MB 00:04 2024-09-16T14:04:04.660 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-09-16T14:04:05.068 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-09-16T14:04:05.068 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-09-16T14:04:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-09-16T14:04:06.076 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-09-16T14:04:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-09-16T14:04:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:04:09.398 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:04:12.200 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:04:12.336 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:04:17.464 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:04:17.467 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.0 MB/s | 35 MB 00:17 2024-09-16T14:04:17.467 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:04:17.467 INFO:teuthology.orchestra.run.smithi189.stdout:Total 4.6 MB/s | 79 MB 00:17 2024-09-16T14:04:17.503 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:04:18.148 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-16T14:04:18.480 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-16T14:04:18.480 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-16T14:04:19.413 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-16T14:04:19.413 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-16T14:04:21.868 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-16T14:04:22.528 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:04:22.609 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:04:22.644 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:04:25.421 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:04:25.638 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:04:30.486 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:04:30.589 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:04:35.436 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:01.596 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:01.637 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:02.032 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:02.032 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:05:02.032 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:05:02.032 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:05:02.879 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-09-16T14:05:03.205 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:05:03.350 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:05:03.350 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:05:03.350 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:05:03.350 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:05:03.350 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:05:03.351 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T14:05:03.351 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:05:03.351 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:05:03.409 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:05:03.409 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:05:03.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-09-16T14:05:03.440 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 82.el9 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8274670 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:05:03.441 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:05:03.442 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:05:03.443 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:05:03.443 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:05:03.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:05:04.053 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-09-16T14:05:05.317 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:05:05.350 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-09-16T14:05:05.352 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-16T14:05:05.367 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.DqkFpcbrvl 2024-09-16T14:05:05.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DqkFpcbrvl 2024-09-16T14:05:05.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.DqkFpcbrvl 2024-09-16T14:05:05.576 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.DqkFpcbrvl is 6KB 2024-09-16T14:05:05.626 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.DqkFpcbrvl 2024-09-16T14:05:05.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:05:05.707 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:05:05.708 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:05:05.708 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:05:05.708 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:05:05.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:05:05.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-09-16T14:05:13.488 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:13.873 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:13.873 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:05:13.874 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:05:13.874 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:05:14.730 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:05:14.730 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:05:14.730 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-16T14:05:14.730 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:05:14.730 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:05:14.731 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:05:14.731 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:05:14.731 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:05:14.731 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-16T14:05:15.081 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:05:15.156 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:05:15.156 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:05:15.157 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:05:15.157 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:05:15.157 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:05:15.157 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T14:05:15.157 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:05:15.157 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:05:15.215 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:05:15.216 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:05:15.216 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-09-16T14:05:15.250 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-09-16T14:05:15.250 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-09-16T14:05:15.250 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 82.el9 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8274670 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:05:15.251 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:05:15.252 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:05:15.253 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:05:15.253 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:05:15.253 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:05:15.905 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-09-16T14:05:17.229 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:05:17.268 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-09-16T14:05:17.271 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-16T14:05:17.287 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.ETHRM2C6qi 2024-09-16T14:05:17.287 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ETHRM2C6qi 2024-09-16T14:05:17.354 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.ETHRM2C6qi 2024-09-16T14:05:17.490 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.ETHRM2C6qi is 6KB 2024-09-16T14:05:17.538 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.ETHRM2C6qi 2024-09-16T14:05:17.553 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:05:17.616 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:05:17.616 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:05:17.616 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:05:17.616 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:05:17.617 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:05:17.781 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-09-16T14:05:35.873 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T14:05:35.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T14:05:35.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:05:47.794 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T14:05:47.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:05:47.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:06:06.145 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-16T14:06:15.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:06:15.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:06:18.237 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-16T14:06:30.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:06:30.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:06:35.883 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T14:06:44.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T14:06:44.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:06:45.583 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-16T14:06:46.095 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T14:06:46.095 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:06:46.095 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-16T14:06:46.110 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:06:46.110 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:06:46.110 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:06:46.110 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:06:47.112 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T14:06:47.112 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:06:47.200 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-09-16T14:06:47.215 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:30.249 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T14:07:45.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:07:45.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:07:45.706 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-16T14:07:46.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:07:46.052 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:07:46.052 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:07:46.068 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:07:46.068 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:07:46.068 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:07:47.069 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T14:07:47.069 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:07:47.170 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-09-16T14:07:47.190 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:47.190 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T14:07:47.199 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T14:07:47.199 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:07:47.202 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:07:47.219 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T14:07:47.241 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T14:07:47.275 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T14:07:47.275 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:07:47.278 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:07:47.309 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T14:07:47.315 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T14:07:47.316 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:07:47.342 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:07:47.374 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:07:47.386 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:07:47.387 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:07:47.399 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:07:47.401 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T14:07:47.412 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T14:07:47.412 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:07:47.430 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:07:47.469 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T14:07:47.478 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T14:07:47.478 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:07:47.498 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:07:47.525 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:07:47.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:07:47.648 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:07:47.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:07:47.776 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:07:47.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:07:47.844 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:07:47.882 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:07:47.946 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:07:48.031 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:07:48.076 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:07:48.076 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:07:48.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-09-16T14:07:48.147 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-16T14:07:48.195 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:07:48.228 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:07:48.625 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T14:07:48.631 INFO:teuthology.task.internal:Starting timer... 2024-09-16T14:07:48.631 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T14:07:48.662 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T14:07:48.694 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-16T14:07:48.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-09-16T14:07:48.749 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-09-16T14:07:48.751 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-16T14:07:48.791 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-16T14:07:48.792 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T14:07:48.793 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-09-16T14:07:48.824 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-09-16T14:07:48.825 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-16T14:07:48.863 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-16T14:07:48.864 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-09-16T14:07:48.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T14:07:48.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:07:48.894 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T14:07:48.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:07:48.932 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T14:07:48.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:07:48.971 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:07:49.008 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T14:07:49.018 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T14:07:49.030 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T14:07:49.031 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T14:11:34.384 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-16T14:11:34.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T14:11:34.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:11:34.465 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-16T14:11:34.541 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T14:11:34.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:11:34.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:11:34.617 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-16T14:11:34.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-16T14:11:34.697 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T14:11:34.708 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T14:11:34.708 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:11:34.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:11:34.711 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:11:34.711 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:11:34.739 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:11:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:11:34.766 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:11:34.779 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-09-16T14:11:34.781 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:11:34.790 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-09-16T14:11:34.805 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-16T14:11:34.805 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:11:34.816 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-16T14:11:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:11:34.831 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:11:34.845 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:11:34.877 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:11:34.881 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:11:34.882 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-09-16T14:11:34.882 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.882 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.882 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.882 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.896 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:11:34.900 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:11:34.900 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-16T14:11:34.900 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.900 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.900 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.901 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:11:34.901 INFO:teuthology.run_tasks:Running task pexec... 2024-09-16T14:11:34.911 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-16T14:11:34.911 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T14:11:34.912 DEBUG:teuthology.orchestra.run.smithi012:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T14:11:34.912 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi189.front.sepia.ceph.com 2024-09-16T14:11:34.912 DEBUG:teuthology.orchestra.run.smithi189:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T14:11:35.289 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-09-16T14:11:35.296 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-16T14:11:35.298 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:11:35.298 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-09-16T14:11:35.298 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:Removing: 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:Removing unused dependencies: 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:Remove 2 Packages 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:Freed space: 5.5 M 2024-09-16T14:11:35.299 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-09-16T14:11:35.303 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-09-16T14:11:35.303 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-09-16T14:11:35.311 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:11:35.311 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-16T14:11:35.311 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:11:35.311 INFO:teuthology.orchestra.run.smithi189.stdout:Removing: 2024-09-16T14:11:35.311 INFO:teuthology.orchestra.run.smithi189.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout:Removing unused dependencies: 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout:Remove 2 Packages 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout:Freed space: 5.5 M 2024-09-16T14:11:35.312 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-16T14:11:35.316 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-16T14:11:35.316 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-16T14:11:35.367 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-09-16T14:11:35.367 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-09-16T14:11:35.463 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-16T14:11:35.463 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-16T14:11:35.518 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-09-16T14:11:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-16T14:11:35.598 INFO:teuthology.orchestra.run.smithi189.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T14:11:35.606 INFO:teuthology.orchestra.run.smithi012.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T14:11:35.655 INFO:teuthology.orchestra.run.smithi189.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T14:11:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T14:11:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T14:11:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T14:11:36.163 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T14:11:36.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T14:11:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T14:11:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:36.308 INFO:teuthology.orchestra.run.smithi189.stdout:Removed: 2024-09-16T14:11:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T14:11:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:36.308 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-16T14:11:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T14:11:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:36.419 INFO:teuthology.orchestra.run.smithi012.stdout:Removed: 2024-09-16T14:11:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T14:11:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:36.419 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-09-16T14:11:37.120 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 02:10:48 PM UTC. 2024-09-16T14:11:37.264 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-16T14:11:37.265 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:11:37.265 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-16T14:11:37.265 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:11:37.265 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-16T14:11:37.265 INFO:teuthology.orchestra.run.smithi189.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T14:11:37.265 INFO:teuthology.orchestra.run.smithi189.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2024-09-16T14:11:37.266 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:37.294 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 2.0 M 2024-09-16T14:11:37.294 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 9.2 M 2024-09-16T14:11:37.294 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-16T14:11:37.355 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:47 ago on Mon 16 Sep 2024 02:10:50 PM UTC. 2024-09-16T14:11:37.444 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 443 kB/s | 44 kB 00:00 2024-09-16T14:11:37.477 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 792 kB/s | 106 kB 00:00 2024-09-16T14:11:37.493 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T14:11:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T14:11:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:37.495 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-09-16T14:11:37.495 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-09-16T14:11:37.495 INFO:teuthology.orchestra.run.smithi012.stdout:Install 6 Packages 2024-09-16T14:11:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:37.496 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 2.0 M 2024-09-16T14:11:37.497 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 9.2 M 2024-09-16T14:11:37.497 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-09-16T14:11:37.519 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 955 kB/s | 72 kB 00:00 2024-09-16T14:11:37.553 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-16T14:11:37.664 INFO:teuthology.orchestra.run.smithi012.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 315 kB/s | 44 kB 00:00 2024-09-16T14:11:37.687 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.6 MB/s | 906 kB 00:00 2024-09-16T14:11:37.689 INFO:teuthology.orchestra.run.smithi012.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 638 kB/s | 106 kB 00:00 2024-09-16T14:11:37.740 INFO:teuthology.orchestra.run.smithi012.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-16T14:11:37.765 INFO:teuthology.orchestra.run.smithi012.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 713 kB/s | 72 kB 00:00 2024-09-16T14:11:37.845 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 2.5 MB/s | 837 kB 00:00 2024-09-16T14:11:37.846 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:11:37.846 INFO:teuthology.orchestra.run.smithi189.stdout:Total 3.6 MB/s | 2.0 MB 00:00 2024-09-16T14:11:37.865 INFO:teuthology.orchestra.run.smithi012.stdout:(5/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 6.5 MB/s | 837 kB 00:00 2024-09-16T14:11:37.917 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-16T14:11:37.930 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-16T14:11:37.930 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-16T14:11:37.957 INFO:teuthology.orchestra.run.smithi012.stdout:(6/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.0 MB/s | 906 kB 00:00 2024-09-16T14:11:37.957 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:11:37.957 INFO:teuthology.orchestra.run.smithi012.stdout:Total 4.3 MB/s | 2.0 MB 00:00 2024-09-16T14:11:38.034 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-09-16T14:11:38.049 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-09-16T14:11:38.049 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-09-16T14:11:38.229 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-16T14:11:38.229 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-16T14:11:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-09-16T14:11:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-09-16T14:11:38.659 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-16T14:11:38.756 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-09-16T14:11:38.757 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-16T14:11:38.791 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-16T14:11:38.827 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-16T14:11:38.828 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-16T14:11:38.862 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-16T14:11:38.890 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-16T14:11:39.041 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-16T14:11:39.075 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-16T14:11:39.147 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-16T14:11:39.179 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-16T14:11:39.511 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-16T14:11:39.545 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-16T14:11:39.654 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-16T14:11:39.681 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-16T14:11:40.531 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-16T14:11:40.532 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-16T14:11:40.532 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-16T14:11:40.532 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-16T14:11:40.532 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-16T14:11:40.532 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-16T14:11:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-16T14:11:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-16T14:11:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-16T14:11:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-16T14:11:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-16T14:11:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:11:40.897 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:11:41.011 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-09-16T14:11:41.182 DEBUG:teuthology.parallel:result is None 2024-09-16T14:11:41.364 DEBUG:teuthology.parallel:result is None 2024-09-16T14:11:41.364 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-16T14:11:41.419 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-16T14:11:41.419 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:11:41.419 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:11:41.436 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:11:41.436 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:11:40.043522997 +0000 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:39.771515562 +0000 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:39.771515562 +0000 2024-09-16T14:11:41.494 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:39.771515562 +0000 2024-09-16T14:11:41.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:11:41.561 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:11:41.561 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:11:41.561 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000268131 s, 1.9 MB/s 2024-09-16T14:11:41.562 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:11:41.619 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:11:40.043522997 +0000 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:39.773515616 +0000 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:39.773515616 +0000 2024-09-16T14:11:41.676 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:39.773515616 +0000 2024-09-16T14:11:41.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:11:41.741 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:11:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:11:41.742 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000165729 s, 3.1 MB/s 2024-09-16T14:11:41.743 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:11:41.798 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-09-16T14:11:41.854 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:11:41.854 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:11:40.044523025 +0000 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:39.773515616 +0000 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:39.773515616 +0000 2024-09-16T14:11:41.855 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:39.773515616 +0000 2024-09-16T14:11:41.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:11:41.918 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:11:41.918 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:11:41.918 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000181655 s, 2.8 MB/s 2024-09-16T14:11:41.919 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:11:41.976 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:11:40.044523025 +0000 2024-09-16T14:11:42.032 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:39.772515589 +0000 2024-09-16T14:11:42.033 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:39.772515589 +0000 2024-09-16T14:11:42.033 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:39.772515589 +0000 2024-09-16T14:11:42.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:11:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:11:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:11:42.096 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000191105 s, 2.7 MB/s 2024-09-16T14:11:42.097 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:11:42.154 DEBUG:teuthology.orchestra.run.smithi012:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-16T14:11:42.297 INFO:teuthology.orchestra.run.smithi012.stdout:loop 2024-09-16T14:11:42.299 INFO:tasks.nvme_loop:Connecting nvme_loop smithi012:/dev/vg_nvme/lv_1... 2024-09-16T14:11:42.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-16T14:11:42.333 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T14:11:42.364 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/vg_nvme/lv_11 2024-09-16T14:11:42.408 INFO:teuthology.orchestra.run.smithi012.stdout:connecting to device: nvme1 2024-09-16T14:11:42.410 INFO:tasks.nvme_loop:Connecting nvme_loop smithi012:/dev/vg_nvme/lv_2... 2024-09-16T14:11:42.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-16T14:11:42.487 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/vg_nvme/lv_21 2024-09-16T14:11:42.549 INFO:teuthology.orchestra.run.smithi012.stdout:connecting to device: nvme2 2024-09-16T14:11:42.551 INFO:tasks.nvme_loop:Connecting nvme_loop smithi012:/dev/vg_nvme/lv_3... 2024-09-16T14:11:42.551 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-16T14:11:42.626 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T14:11:42.657 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/vg_nvme/lv_31 2024-09-16T14:11:42.688 INFO:teuthology.orchestra.run.smithi012.stdout:connecting to device: nvme3 2024-09-16T14:11:42.690 INFO:tasks.nvme_loop:Connecting nvme_loop smithi012:/dev/vg_nvme/lv_4... 2024-09-16T14:11:42.690 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-16T14:11:42.767 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T14:11:42.798 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/vg_nvme/lv_41 2024-09-16T14:11:42.830 INFO:teuthology.orchestra.run.smithi012.stdout:connecting to device: nvme4 2024-09-16T14:11:42.832 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:11:42.832 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:11:42.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo nvme list -o json 2024-09-16T14:11:42.954 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "Devices":[ 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "NameSpace":1, 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "GenericPath":"/dev/ng4n1", 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "SerialNumber":"84776b20176c1979cb3b", 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "SectorSize":512 2024-09-16T14:11:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "NameSpace":1, 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "GenericPath":"/dev/ng3n1", 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "SerialNumber":"2c380defbffe8e1f4fb0", 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "SectorSize":512 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T14:11:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "NameSpace":1, 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "GenericPath":"/dev/ng2n1", 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "SerialNumber":"9333e8110a13490cf305", 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "SectorSize":512 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "NameSpace":1, 2024-09-16T14:11:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "GenericPath":"/dev/ng1n1", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "SerialNumber":"b0914b709fa4950f04ca", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "SectorSize":512 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "NameSpace":1, 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "GenericPath":"/dev/ng0n1", 2024-09-16T14:11:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "Firmware":"8DV101H0", 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "SerialNumber":"CVFT5331005J400BGN", 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "UsedBytes":400088457216, 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "MaximumLBA":781422768, 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "PhysicalSize":400088457216, 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "SectorSize":512 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T14:11:42.959 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T14:11:42.960 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T14:11:42.960 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:11:42.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/scratch_devs 2024-09-16T14:11:43.026 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:11:43.026 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:11:43.046 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:11:43.046 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-16T14:11:43.105 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:11:43.105 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:43.105 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-16T14:11:43.105 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:43.105 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:43.106 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:11:39.855410802 +0000 2024-09-16T14:11:43.106 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.106 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.106 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.106 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:11:43.175 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:11:43.175 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:11:43.175 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000182357 s, 2.8 MB/s 2024-09-16T14:11:43.177 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:11:43.236 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:11:39.855410802 +0000 2024-09-16T14:11:43.294 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:39.629410351 +0000 2024-09-16T14:11:43.295 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:39.629410351 +0000 2024-09-16T14:11:43.295 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:39.629410351 +0000 2024-09-16T14:11:43.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:11:43.359 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:11:43.360 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:11:43.360 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000201796 s, 2.5 MB/s 2024-09-16T14:11:43.361 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:11:43.418 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-16T14:11:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:11:39.856410833 +0000 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.475 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:11:43.539 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:11:43.539 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:11:43.540 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000215361 s, 2.4 MB/s 2024-09-16T14:11:43.541 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:11:43.597 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-16T14:11:43.653 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:11:39.856410833 +0000 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.654 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:39.627410390 +0000 2024-09-16T14:11:43.654 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:11:43.718 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:11:43.718 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:11:43.718 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000188485 s, 2.7 MB/s 2024-09-16T14:11:43.719 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:11:43.775 DEBUG:teuthology.orchestra.run.smithi189:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-16T14:11:43.912 INFO:teuthology.orchestra.run.smithi189.stdout:loop 2024-09-16T14:11:43.913 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_1... 2024-09-16T14:11:43.913 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-16T14:11:43.946 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-16T14:11:43.976 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_11 2024-09-16T14:11:44.012 INFO:teuthology.orchestra.run.smithi189.stdout:connecting to device: nvme1 2024-09-16T14:11:44.014 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_2... 2024-09-16T14:11:44.014 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-16T14:11:44.088 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-16T14:11:44.135 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_21 2024-09-16T14:11:44.171 INFO:teuthology.orchestra.run.smithi189.stdout:connecting to device: nvme2 2024-09-16T14:11:44.173 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_3... 2024-09-16T14:11:44.173 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-16T14:11:44.248 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-16T14:11:44.278 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_31 2024-09-16T14:11:44.309 INFO:teuthology.orchestra.run.smithi189.stdout:connecting to device: nvme3 2024-09-16T14:11:44.311 INFO:tasks.nvme_loop:Connecting nvme_loop smithi189:/dev/vg_nvme/lv_4... 2024-09-16T14:11:44.311 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-16T14:11:44.385 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-16T14:11:44.416 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/vg_nvme/lv_41 2024-09-16T14:11:44.451 INFO:teuthology.orchestra.run.smithi189.stdout:connecting to device: nvme4 2024-09-16T14:11:44.453 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:11:44.454 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:11:44.511 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme list -o json 2024-09-16T14:11:44.577 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-16T14:11:44.577 INFO:teuthology.orchestra.run.smithi189.stdout: "Devices":[ 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "NameSpace":1, 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "GenericPath":"/dev/ng4n1", 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "SerialNumber":"b535c61445c2800adb5b", 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: "SectorSize":512 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-16T14:11:44.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "NameSpace":1, 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "GenericPath":"/dev/ng3n1", 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "SerialNumber":"e9a18bb23920b2aaeff1", 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "SectorSize":512 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "NameSpace":1, 2024-09-16T14:11:44.579 INFO:teuthology.orchestra.run.smithi189.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "GenericPath":"/dev/ng2n1", 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "SerialNumber":"94773ca22734a2a60893", 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "SectorSize":512 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "NameSpace":1, 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-16T14:11:44.580 INFO:teuthology.orchestra.run.smithi189.stdout: "GenericPath":"/dev/ng1n1", 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "Firmware":"5.14.0-5", 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "ModelNumber":"Linux", 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "SerialNumber":"abba6dffe417aee506d0", 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "UsedBytes":95995035648, 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "MaximumLBA":187490304, 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "PhysicalSize":95995035648, 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "SectorSize":512 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-16T14:11:44.581 INFO:teuthology.orchestra.run.smithi189.stdout: "NameSpace":1, 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "GenericPath":"/dev/ng0n1", 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "Firmware":"8DV101H0", 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "SerialNumber":"CVFT6232004L400BGN", 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "UsedBytes":400088457216, 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "MaximumLBA":781422768, 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "PhysicalSize":400088457216, 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: "SectorSize":512 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-16T14:11:44.582 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-16T14:11:44.583 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T14:11:44.583 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:11:44.583 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/scratch_devs 2024-09-16T14:11:44.648 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T14:11:44.734 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'CEPHADM_AGENT_DOWN'], 'log-only-match': ['CEPHADM_'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T14:11:44.734 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:11:44.735 INFO:tasks.cephadm:Cluster fsid is 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:11:44.735 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T14:11:44.735 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.c': '[v2:172.21.15.12:3301,v1:172.21.15.12:6790]', 'mon.b': '172.21.15.189'} 2024-09-16T14:11:44.735 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-09-16T14:11:44.735 INFO:tasks.cephadm:First mgr is y 2024-09-16T14:11:44.735 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T14:11:44.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-09-16T14:11:44.767 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-09-16T14:11:44.793 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T14:11:44.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:11:44.934 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56072/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-845-ge4e227b3', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-845.ge4e227b3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 17:59:03.516496', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T14:11:45.091 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 e4e227b37858f5109138494e01e5ba1ba7b8ffe8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:11:45.092 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm 2024-09-16T14:11:45.092 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T14:11:45.499 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 16 14:11 /home/ubuntu/cephtest/cephadm 2024-09-16T14:11:45.499 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T14:11:45.853 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 16 14:11 /home/ubuntu/cephtest/cephadm 2024-09-16T14:11:45.853 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T14:11:45.873 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T14:11:45.901 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 on all hosts... 2024-09-16T14:11:45.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 pull 2024-09-16T14:11:45.916 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 pull 2024-09-16T14:11:47.436 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T14:11:47.520 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T14:12:14.373 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T14:12:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T14:12:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1", 2024-09-16T14:12:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-09-16T14:12:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322" 2024-09-16T14:12:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T14:12:14.374 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T14:12:15.124 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-16T14:12:15.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T14:12:15.124 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1", 2024-09-16T14:12:15.125 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-09-16T14:12:15.125 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322" 2024-09-16T14:12:15.125 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-16T14:12:15.125 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-16T14:12:15.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-09-16T14:12:15.177 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-09-16T14:12:15.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-09-16T14:12:15.247 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-09-16T14:12:15.272 INFO:tasks.cephadm:Writing seed config... 2024-09-16T14:12:15.274 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T14:12:15.274 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T14:12:15.274 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T14:12:15.275 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T14:12:15.275 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:12:15.276 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T14:12:15.306 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9a9578d0-7435-11ef-bceb-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-16T14:12:15.307 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T14:12:15.349 DEBUG:teuthology.orchestra.run.smithi012:mgr.y> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T14:12:15.392 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T14:12:15.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 -v bootstrap --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:12:15.554 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:12:15.554 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8', '-v', 'bootstrap', '--fsid', '9a9578d0-7435-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-09-16T14:12:15.579 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.2.2 2024-09-16T14:12:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T14:12:15.579 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-09-16T14:12:15.602 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.2.2 2024-09-16T14:12:15.602 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-09-16T14:12:15.602 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-09-16T14:12:15.609 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T14:12:15.609 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T14:12:15.616 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T14:12:15.616 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-16T14:12:15.623 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-09-16T14:12:15.630 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-09-16T14:12:15.630 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-09-16T14:12:15.630 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-09-16T14:12:15.652 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.2.2 2024-09-16T14:12:15.652 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T14:12:15.653 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-09-16T14:12:15.653 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-09-16T14:12:15.661 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T14:12:15.661 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T14:12:15.666 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T14:12:15.666 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-16T14:12:15.673 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-09-16T14:12:15.680 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-09-16T14:12:15.680 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-09-16T14:12:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-09-16T14:12:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 140372901908640 on /run/cephadm/9a9578d0-7435-11ef-bceb-c7b262605968.lock 2024-09-16T14:12:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 140372901908640 acquired on /run/cephadm/9a9578d0-7435-11ef-bceb-c7b262605968.lock 2024-09-16T14:12:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-09-16T14:12:15.681 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-09-16T14:12:15.682 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-09-16T14:12:15.684 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-09-16T14:12:15.684 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-09-16T14:12:15.686 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T14:12:15.687 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-16T14:12:15.687 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1761sec hoplimit 64 pref medium 2024-09-16T14:12:15.690 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T14:12:15.690 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T14:12:15.690 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T14:12:15.690 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-16T14:12:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-09-16T14:12:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T14:12:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-09-16T14:12:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-09-16T14:12:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T14:12:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T14:12:15.692 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T14:12:16.159 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T14:12:16.159 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T14:12:16.159 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T14:12:16.159 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T14:12:16.159 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:71878612010b3dfa9e3e456d432a49e20ac26634831a8a6522f4d782181c5840 2024-09-16T14:12:16.159 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T14:12:16.160 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T14:12:17.046 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T14:12:17.046 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T14:12:17.046 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T14:12:17.985 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-09-16T14:12:17.985 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-09-16T14:12:18.998 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQDCPOhmSsflERAAWcvzpv/EbMvXkDtIWCzSKQ== 2024-09-16T14:12:19.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQDDPOhmHUQpEhAAomDDcm9Wc/k0/OOeF0XFBQ== 2024-09-16T14:12:21.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQDEPOhmL1yeERAALnQg90hVd/TBsIJ2J75Ecw== 2024-09-16T14:12:21.126 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:12:22.041 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-09-16T14:12:22.920 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-09-16T14:12:23.320 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T14:12:23.510 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9a9578d0-7435-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9a9578d0-7435-11ef-bceb-c7b262605968.target. 2024-09-16T14:12:23.511 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9a9578d0-7435-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9a9578d0-7435-11ef-bceb-c7b262605968.target. 2024-09-16T14:12:23.775 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a 2024-09-16T14:12:23.776 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T14:12:23.966 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9a9578d0-7435-11ef-bceb-c7b262605968.target.wants/ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9a9578d0-7435-11ef-bceb-c7b262605968@.service. 2024-09-16T14:12:24.531 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T14:12:24.531 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-16T14:12:24.540 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T14:12:24.540 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-16T14:12:24.540 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-16T14:12:24.540 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T14:12:24.540 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-09-16T14:12:24.540 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-09-16T14:12:25.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:25 smithi012 ceph-mon[25157]: from='client.? 172.21.15.12:0/4165299514' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T14:12:26.251 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T14:12:26.251 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:26.251 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T14:12:26.251 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:26.251 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.515833s) 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-09-16T14:12:26.252 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T14:12:29.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:29 smithi012 ceph-mon[25157]: from='client.? 172.21.15.12:0/4167905235' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T14:12:29.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:29 smithi012 ceph-mon[25157]: from='client.? 172.21.15.12:0/4167905235' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T14:12:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T14:12:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-09-16T14:12:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T14:12:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T14:12:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-09-16T14:12:38.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:38 smithi012 ceph-mon[25157]: from='client.? 172.21.15.12:0/1827483046' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:12:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-09-16T14:12:38.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:38 smithi012 systemd[1]: Stopping Ceph mon.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:12:39.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:38 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a[25153]: 2024-09-16T14:12:38.861+0000 7f4582cf1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:12:39.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:38 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a[25153]: 2024-09-16T14:12:38.861+0000 7f4582cf1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T14:12:39.171 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:38 smithi012 podman[25402]: 2024-09-16 14:12:38.964599437 +0000 UTC m=+0.219997006 container died cfba4eba7a893eb1b026eaf496f084609d95f3d8b9d3e69481a7678b3f7d719d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:12:39.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 podman[25402]: 2024-09-16 14:12:39.261298779 +0000 UTC m=+0.516696387 container cleanup cfba4eba7a893eb1b026eaf496f084609d95f3d8b9d3e69481a7678b3f7d719d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T14:12:39.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 bash[25402]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a 2024-09-16T14:12:39.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 podman[25417]: 2024-09-16 14:12:39.487152982 +0000 UTC m=+0.522300600 container remove cfba4eba7a893eb1b026eaf496f084609d95f3d8b9d3e69481a7678b3f7d719d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T14:12:39.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T14:12:39.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 systemd[1]: Stopped Ceph mon.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:12:39.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 systemd[1]: Starting Ceph mon.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:12:40.103 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 podman[25509]: 2024-09-16 14:12:39.753780772 +0000 UTC m=+0.016328143 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:39 smithi012 podman[25509]: 2024-09-16 14:12:39.87963922 +0000 UTC m=+0.142186627 container create de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 podman[25509]: 2024-09-16 14:12:40.073303484 +0000 UTC m=+0.335850873 container init de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 podman[25509]: 2024-09-16 14:12:40.076499082 +0000 UTC m=+0.339046446 container start de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 2 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: pidfile_write: ignore empty --pid-file 2024-09-16T14:12:40.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: load: jerasure load: lrc 2024-09-16T14:12:40.162 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T14:12:40.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: RocksDB version: 7.9.2 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Git sha 0 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: DB SUMMARY 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: DB Session ID: HIPBJEXPBS8XLVUJE71W 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: CURRENT file: CURRENT 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T14:12:40.419 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83738 ; 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.error_if_exists: 0 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.create_if_missing: 0 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.paranoid_checks: 1 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.env: 0x5588ce153bc0 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.info_log: 0x5588cf422be0 2024-09-16T14:12:40.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.statistics: (nil) 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.use_fsync: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_log_file_size: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.allow_fallocate: 1 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.use_direct_reads: 0 2024-09-16T14:12:40.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.db_log_dir: 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.wal_dir: 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T14:12:40.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T14:12:40.424 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.write_buffer_manager: 0x5588cf430320 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T14:12:40.425 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.unordered_write: 0 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.row_cache: None 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.wal_filter: None 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T14:12:40.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.two_write_queues: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.wal_compression: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.atomic_flush: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.log_readahead_size: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T14:12:40.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_background_jobs: 2 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_background_compactions: -1 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_subcompactions: 1 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T14:12:40.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_open_files: -1 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_background_flushes: -1 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Compression algorithms supported: 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kZSTD supported: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kXpressCompression supported: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kBZip2Compression supported: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T14:12:40.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kLZ4Compression supported: 1 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kZlibCompression supported: 1 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: kSnappyCompression supported: 1 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.merge_operator: 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_filter: None 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T14:12:40.430 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5588cf4227e0) 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x5588cf417350 2024-09-16T14:12:40.431 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-09-16T14:12:40.432 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T14:12:40.433 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression: NoCompression 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.num_levels: 7 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T14:12:40.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T14:12:40.435 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T14:12:40.436 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T14:12:40.436 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T14:12:40.436 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T14:12:40.436 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T14:12:40.436 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T14:12:40.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T14:12:40.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T14:12:40.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.inplace_update_support: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.bloom_locality: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.max_successive_merges: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T14:12:40.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.ttl: 2592000 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.enable_blob_files: false 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.min_blob_size: 0 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T14:12:40.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eb57d02c-43d6-46fa-8141-a262a11bf2b7 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726495960104904, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726495960105782, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80511, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78672, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 72980, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726495960, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eb57d02c-43d6-46fa-8141-a262a11bf2b7", "db_session_id": "HIPBJEXPBS8XLVUJE71W", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726495960105858, "job": 1, "event": "recovery_finished"} 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5588cf442e00 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: DB pointer 0x5588cf536000 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T14:12:40.442 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 80.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T14:12:40.443 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 80.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T14:12:40.444 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 24.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 24.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x5588cf417350#2 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???) e1 preinit fsid 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:40.445 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).mds e1 new map 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).mds e1 print_map 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 bash[25509]: de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 systemd[1]: Started Ceph mon.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:12:40.446 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T14:12:40.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: monmap e1: 1 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:12:40.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: fsmap 2024-09-16T14:12:40.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T14:12:40.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:40 smithi012 ceph-mon[25528]: mgrmap e1: no daemons active 2024-09-16T14:12:41.184 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T14:12:41.196 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:12:41.196 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-09-16T14:12:41.197 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T14:12:41.197 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T14:12:41.457 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y 2024-09-16T14:12:41.457 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-16T14:12:41.616 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9a9578d0-7435-11ef-bceb-c7b262605968.target.wants/ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-9a9578d0-7435-11ef-bceb-c7b262605968@.service. 2024-09-16T14:12:41.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:41 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/436701305' entity='client.admin' 2024-09-16T14:12:42.168 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 podman[25739]: 2024-09-16 14:12:42.086400605 +0000 UTC m=+0.348550059 container start bd4030ec6967ecdda51291404d2113f2455822ea561a5f76a9569e29cf457625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T14:12:42.168 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 bash[25739]: bd4030ec6967ecdda51291404d2113f2455822ea561a5f76a9569e29cf457625 2024-09-16T14:12:42.182 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T14:12:42.182 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-16T14:12:42.190 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T14:12:42.190 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-16T14:12:42.190 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-16T14:12:42.190 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T14:12:42.199 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T14:12:42.199 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-16T14:12:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T14:12:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-16T14:12:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-16T14:12:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T14:12:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-09-16T14:12:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-09-16T14:12:42.431 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 systemd[1]: Started Ceph mgr.y for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:12:42.431 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:42.209+0000 7eff482bb040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:12:42.431 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:42.271+0000 7eff482bb040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:12:42.431 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:42.429+0000 7eff482bb040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:12:42.766 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:42.499+0000 7eff482bb040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:12:42.767 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:42.675+0000 7eff482bb040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:12:43.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2903754061' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T14:12:43.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:42.865+0000 7eff482bb040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:12:43.448 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:43 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:43.192+0000 7eff482bb040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:12:43.448 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:43 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:43.255+0000 7eff482bb040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:12:43.448 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:43 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:43.446+0000 7eff482bb040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:12:44.520 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:44.363+0000 7eff482bb040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:12:44.521 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:44.432+0000 7eff482bb040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:12:44.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:44.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:12:44.525 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "9a9578d0-7435-11ef-bceb-c7b262605968", 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:44.526 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T14:12:44.527 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T14:12:44.531 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T14:12:44.532 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T14:12:24.490381+0000", 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:12:44.533 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-09-16T14:12:45.456 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.187+0000 7eff482bb040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:12:45.456 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.306+0000 7eff482bb040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:12:45.742 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.453+0000 7eff482bb040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:12:45.742 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.516+0000 7eff482bb040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:12:45.742 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.668+0000 7eff482bb040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:12:45.742 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.739+0000 7eff482bb040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:12:46.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.801+0000 7eff482bb040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:12:46.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:45 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:45.963+0000 7eff482bb040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:12:46.310 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:46.031+0000 7eff482bb040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:12:46.310 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:46.099+0000 7eff482bb040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:12:46.310 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:46.185+0000 7eff482bb040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:12:46.310 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:46.246+0000 7eff482bb040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:12:46.310 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:46.307+0000 7eff482bb040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:12:47.268 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:47 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3599986826' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T14:12:47.268 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:47 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:47.145+0000 7eff482bb040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:12:47.534 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:47.534 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "9a9578d0-7435-11ef-bceb-c7b262605968", 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T14:12:47.535 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T14:12:47.536 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T14:12:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T14:12:47.538 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T14:12:24.490381+0000", 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:12:47.539 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-09-16T14:12:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: Activating manager daemon y 2024-09-16T14:12:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: mgrmap e2: y(active, starting, since 0.00313191s) 2024-09-16T14:12:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T14:12:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T14:12:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T14:12:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: Manager daemon y is now available 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' 2024-09-16T14:12:48.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:48 smithi012 ceph-mon[25528]: from='mgr.14100 172.21.15.12:0/1376121254' entity='mgr.y' 2024-09-16T14:12:49.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:49 smithi012 ceph-mon[25528]: mgrmap e3: y(active, since 1.00737s) 2024-09-16T14:12:50.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:50 smithi012 ceph-mon[25528]: mgrmap e4: y(active, since 2s) 2024-09-16T14:12:50.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:50.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:12:50.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "9a9578d0-7435-11ef-bceb-c7b262605968", 2024-09-16T14:12:50.849 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T14:12:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T14:12:50.851 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.852 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-16T14:12:50.853 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T14:12:24.490381+0000", 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:12:50.854 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-09-16T14:12:51.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:51 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3071792424' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T14:12:51.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:51.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T14:12:51.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:12:51.891 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T14:12:51.892 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-09-16T14:12:52.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:52 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/18601541' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T14:12:52.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:52 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/18601541' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T14:12:53.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3717447978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T14:12:53.468 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ignoring --setuser ceph since I am not root 2024-09-16T14:12:53.468 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ignoring --setgroup ceph since I am not root 2024-09-16T14:12:53.468 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:53.317+0000 7f6630ad0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:12:53.468 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:53.375+0000 7f6630ad0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:12:53.763 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:53.531+0000 7f6630ad0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:12:53.763 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:53.600+0000 7f6630ad0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:12:53.763 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:53.761+0000 7f6630ad0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:12:54.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:53 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:53.907+0000 7f6630ad0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:12:54.301 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:54 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3717447978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T14:12:54.301 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:54 smithi012 ceph-mon[25528]: mgrmap e5: y(active, since 6s) 2024-09-16T14:12:54.301 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:54 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:54.236+0000 7f6630ad0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:12:54.708 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:54 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:54.299+0000 7f6630ad0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:12:54.708 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:54 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:54.493+0000 7f6630ad0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:12:54.988 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:12:54.988 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T14:12:54.988 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T14:12:54.988 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T14:12:54.989 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T14:12:54.989 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:12:54.989 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-09-16T14:12:54.989 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-09-16T14:12:55.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:55 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3300155859' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T14:12:55.691 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:55 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:55.416+0000 7f6630ad0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:12:55.692 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:55 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:55.488+0000 7f6630ad0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:12:56.518 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.246+0000 7f6630ad0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:12:56.518 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.366+0000 7f6630ad0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:12:56.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.516+0000 7f6630ad0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:12:56.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.579+0000 7f6630ad0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:12:56.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.732+0000 7f6630ad0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:12:57.096 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.802+0000 7f6630ad0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:12:57.097 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:56 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:56.864+0000 7f6630ad0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:12:57.097 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:57.027+0000 7f6630ad0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:12:57.097 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:57.094+0000 7f6630ad0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:12:57.373 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:57.163+0000 7f6630ad0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:12:57.374 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:57.249+0000 7f6630ad0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:12:57.374 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:57.310+0000 7f6630ad0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:12:57.374 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:57.371+0000 7f6630ad0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:12:58.182+0000 7f6630ad0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: Active manager daemon y restarted 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: Activating manager daemon y 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: mgrmap e6: y(active, starting, since 0.0030309s) 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T14:12:58.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: Manager daemon y is now available 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: Found migration_current of "None". Setting to last migration. 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:12:58.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:58 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:12:59.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:59 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:12:59.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:59 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:12:59.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:59 smithi012 ceph-mon[25528]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T14:12:59.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:59 smithi012 ceph-mon[25528]: mgrmap e7: y(active, since 1.00538s) 2024-09-16T14:12:59.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:12:59 smithi012 ceph-mon[25528]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T14:12:59.775 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:12:59.775 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T14:12:59.775 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T14:12:59.775 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:12:59.775 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-09-16T14:12:59.776 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:00.327+0000 7f659dffb640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:14:13:00] ENGINE Error in HTTPServer.serve 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: Traceback (most recent call last): 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._connections.run(self.expiration_interval) 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._run(expiration_interval) 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:13:00.402 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: s = self.context.wrap_socket( 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: return self.sslsocket_class._create( 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self.do_handshake() 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._sslobj.do_handshake() 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:13:00.403 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:01.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:00] ENGINE Bus STARTING 2024-09-16T14:13:01.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:01.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:01.003 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:00 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:02.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:01 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:00] ENGINE Serving on https://172.21.15.12:7150 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:01 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:00] ENGINE Error in HTTPServer.serve 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: Traceback (most recent call last): 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._connections.run(self.expiration_interval) 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._run(expiration_interval) 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:13:02.026 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: s = self.context.wrap_socket( 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: return self.sslsocket_class._create( 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: self.do_handshake() 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._sslobj.do_handshake() 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:01 smithi012 ceph-mon[25528]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:01 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:00] ENGINE Serving on http://172.21.15.12:8765 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:01 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:00] ENGINE Bus STARTED 2024-09-16T14:13:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:01 smithi012 ceph-mon[25528]: mgrmap e8: y(active, since 2s) 2024-09-16T14:13:02.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T14:13:02.028 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-09-16T14:13:03.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-mon[25528]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:03.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: Generating public/private rsa key pair. 2024-09-16T14:13:03.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: Your identification has been saved in /tmp/tmpt2qh7qae/key 2024-09-16T14:13:03.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: Your public key has been saved in /tmp/tmpt2qh7qae/key.pub 2024-09-16T14:13:03.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: The key fingerprint is: 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: SHA256:/SGXwtKf4yX1BA0ekQTIflgJLxIw89dFhMePUm1FrP4 ceph-9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: The key's randomart image is: 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: +---[RSA 3072]----+ 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | +....o.OO*+| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | + .oo+o=++| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | o.ooooo=.| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | *o.o.o..| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | S *.+o. .| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | . * +.o | 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | * ...| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | . + E| 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: | . | 2024-09-16T14:13:03.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:02 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: +----[SHA256]-----+ 2024-09-16T14:13:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:03 smithi012 ceph-mon[25528]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:03 smithi012 ceph-mon[25528]: Generating ssh key... 2024-09-16T14:13:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:03 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:03 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:04.690 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbwJm9Pzi/ADG4K//ZEujUv+ALSdp7UwOjkR5+d8zfiXFE0KqNbTfaBZUJFU6Ga7kb3UHHWL560tmJlGnaxWPgSCa+RaJA0mx9r1DHVM/Whnz1Qtt+vGIbdjf255agDYxOTnPSlobJgq1/qcnpXQRA/kVKVZd9X5GA1cyxY+y8DE07raOiW3RCc5Y5XWQsQ9bFlk0xHsnbsIe6C83v8V0j/EkCSj7oFuuMRZ6IM99wOwc/Z5GmLbBeHgNV5ar4nExL7WT0kmYfOtaxB5y9fJGlFeA1DOj8omzQjB7aKUjpWyZWt8DFui4k+sZZFgPTTrHUdu1bP9zSR7t74uv5pmaMJOqccQ0aXxT/kuRDgIW13kOLOsA9djU8hAiy1szr25E6zwhxJ18cAEmoZTcwRYqIBCKxxAIwehLCG/96IhcaR1ifH8sykAp5NA1rzJg6eAKy+0C2M6l6I3/lRr+xtiTWolBzpeZ/pcRErFpffO2Adjg7+Rv2TI+Ye7WWSSxBIbk= ceph-9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:13:04.690 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T14:13:04.690 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T14:13:04.698 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-09-16T14:13:04.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:04 smithi012 ceph-mon[25528]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:06.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:06 smithi012 ceph-mon[25528]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:07.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:07 smithi012 ceph-mon[25528]: Deploying cephadm binary to smithi012 2024-09-16T14:13:09.163 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-09-16T14:13:09.164 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-09-16T14:13:09.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:09 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:09.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:09 smithi012 ceph-mon[25528]: Added host smithi012 2024-09-16T14:13:09.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:09 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:10.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T14:13:10.781 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-09-16T14:13:11.106 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:10 smithi012 ceph-mon[25528]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:11.106 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:10 smithi012 ceph-mon[25528]: Saving service mon spec with placement count:5 2024-09-16T14:13:11.106 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:10 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:12.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T14:13:12.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:12 smithi012 ceph-mon[25528]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:12.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:12 smithi012 ceph-mon[25528]: Saving service mgr spec with placement count:2 2024-09-16T14:13:12.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:12 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:12.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:12 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:13.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:13 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:13.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:13 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3800712902' entity='client.admin' 2024-09-16T14:13:14.639 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-09-16T14:13:15.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:14 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1701710305' entity='client.admin' 2024-09-16T14:13:16.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:15 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:16.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:15 smithi012 ceph-mon[25528]: from='mgr.14118 172.21.15.12:0/4041919457' entity='mgr.y' 2024-09-16T14:13:16.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:15 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1052422426' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T14:13:16.971 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:16 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ignoring --setuser ceph since I am not root 2024-09-16T14:13:16.971 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:16 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ignoring --setgroup ceph since I am not root 2024-09-16T14:13:16.971 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:16 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:16.756+0000 7f78ef9ce040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:13:16.971 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:16 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:16.815+0000 7f78ef9ce040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:13:16.971 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:16 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:16.969+0000 7f78ef9ce040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:13:17.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:17.037+0000 7f78ef9ce040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:13:17.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:17.190+0000 7f78ef9ce040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:13:17.597 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:17.336+0000 7f78ef9ce040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:13:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1052422426' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T14:13:17.914 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-mon[25528]: mgrmap e9: y(active, since 18s) 2024-09-16T14:13:17.914 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:17.659+0000 7f78ef9ce040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:13:17.914 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:17.727+0000 7f78ef9ce040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:13:18.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:17.923+0000 7f78ef9ce040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:13:18.852 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:18 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/592310695' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-09-16T14:13:18.970 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-09-16T14:13:19.205 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:18 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:18.850+0000 7f78ef9ce040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:13:19.205 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:18 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:18.919+0000 7f78ef9ce040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:13:19.946 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:19 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:19.677+0000 7f78ef9ce040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:13:19.946 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:19 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:19.797+0000 7f78ef9ce040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:13:20.232 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:19 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:19.944+0000 7f78ef9ce040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:13:20.233 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.007+0000 7f78ef9ce040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:13:20.233 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.160+0000 7f78ef9ce040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:13:20.233 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.230+0000 7f78ef9ce040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:13:20.525 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.293+0000 7f78ef9ce040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:13:20.525 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.455+0000 7f78ef9ce040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:13:20.525 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.523+0000 7f78ef9ce040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:13:20.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.591+0000 7f78ef9ce040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:13:20.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.677+0000 7f78ef9ce040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:13:20.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.739+0000 7f78ef9ce040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:13:21.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:20 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:20.800+0000 7f78ef9ce040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:13:21.868 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:21.611+0000 7f78ef9ce040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: Active manager daemon y restarted 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: Activating manager daemon y 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: mgrmap e10: y(active, starting, since 0.00354238s) 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T14:13:21.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: Manager daemon y is now available 2024-09-16T14:13:21.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:21.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:13:21.870 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:13:23.618 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:23.564+0000 7f7855e58640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:14:13:23] ENGINE Error in HTTPServer.serve 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: Traceback (most recent call last): 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._connections.run(self.expiration_interval) 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._run(expiration_interval) 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:13:23.619 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: s = self.context.wrap_socket( 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: return self.sslsocket_class._create( 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self.do_handshake() 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._sslobj.do_handshake() 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:13:23.620 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:13:23.621 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-16T14:13:23.621 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T14:13:23.621 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T14:13:23.621 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-16T14:13:23.621 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-09-16T14:13:23.622 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-09-16T14:13:23.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-mon[25528]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T14:13:23.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-mon[25528]: mgrmap e11: y(active, since 1.00592s) 2024-09-16T14:13:23.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-mon[25528]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T14:13:23.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:24.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:23] ENGINE Bus STARTING 2024-09-16T14:13:24.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:23] ENGINE Serving on http://172.21.15.12:8765 2024-09-16T14:13:24.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:23] ENGINE Serving on https://172.21.15.12:7150 2024-09-16T14:13:24.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:23] ENGINE Bus STARTED 2024-09-16T14:13:24.842 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: [16/Sep/2024:14:13:23] ENGINE Error in HTTPServer.serve 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: Traceback (most recent call last): 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._connections.run(self.expiration_interval) 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._run(expiration_interval) 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: s = self.context.wrap_socket( 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:13:24.843 INFO:journalctl@ceph.mon.a.smithi012.stdout: return self.sslsocket_class._create( 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout: self.do_handshake() 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._sslobj.do_handshake() 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: mgrmap e12: y(active, since 2s) 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:24.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:24 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:24.845 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T14:13:24.845 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-09-16T14:13:25.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:25 smithi012 ceph-mon[25528]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:25.865 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:25 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:26.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:26 smithi012 ceph-mon[25528]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:26.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:27.199 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$zbirGAu/HUUAP5prbrG6fe.HTB.My.PLVxiEgP6TAUq8mMCpP/J/W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726496005, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T14:13:27.199 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-09-16T14:13:28.331 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T14:13:28.342 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T14:13:28.342 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-16T14:13:28.350 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T14:13:28.350 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-16T14:13:28.350 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-16T14:13:28.350 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T14:13:28.352 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-09-16T14:13:28.352 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:28.352 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-09-16T14:13:28.352 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-09-16T14:13:28.353 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 7nqjd1vjal 2024-09-16T14:13:28.353 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:28.353 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config directory 2024-09-16T14:13:28.356 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-09-16T14:13:28.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:28 smithi012 ceph-mon[25528]: mgrmap e13: y(active, since 6s) 2024-09-16T14:13:28.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:28 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2634866356' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.778 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:30.779 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-09-16T14:13:30.816 INFO:tasks.cephadm:Fetching config... 2024-09-16T14:13:30.817 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:13:30.817 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T14:13:30.835 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T14:13:30.835 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:13:30.835 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T14:13:30.892 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T14:13:30.893 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:13:30.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T14:13:30.959 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T14:13:30.959 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:13:30.959 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T14:13:31.016 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T14:13:31.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbwJm9Pzi/ADG4K//ZEujUv+ALSdp7UwOjkR5+d8zfiXFE0KqNbTfaBZUJFU6Ga7kb3UHHWL560tmJlGnaxWPgSCa+RaJA0mx9r1DHVM/Whnz1Qtt+vGIbdjf255agDYxOTnPSlobJgq1/qcnpXQRA/kVKVZd9X5GA1cyxY+y8DE07raOiW3RCc5Y5XWQsQ9bFlk0xHsnbsIe6C83v8V0j/EkCSj7oFuuMRZ6IM99wOwc/Z5GmLbBeHgNV5ar4nExL7WT0kmYfOtaxB5y9fJGlFeA1DOj8omzQjB7aKUjpWyZWt8DFui4k+sZZFgPTTrHUdu1bP9zSR7t74uv5pmaMJOqccQ0aXxT/kuRDgIW13kOLOsA9djU8hAiy1szr25E6zwhxJ18cAEmoZTcwRYqIBCKxxAIwehLCG/96IhcaR1ifH8sykAp5NA1rzJg6eAKy+0C2M6l6I3/lRr+xtiTWolBzpeZ/pcRErFpffO2Adjg7+Rv2TI+Ye7WWSSxBIbk= ceph-9a9578d0-7435-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T14:13:31.094 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbwJm9Pzi/ADG4K//ZEujUv+ALSdp7UwOjkR5+d8zfiXFE0KqNbTfaBZUJFU6Ga7kb3UHHWL560tmJlGnaxWPgSCa+RaJA0mx9r1DHVM/Whnz1Qtt+vGIbdjf255agDYxOTnPSlobJgq1/qcnpXQRA/kVKVZd9X5GA1cyxY+y8DE07raOiW3RCc5Y5XWQsQ9bFlk0xHsnbsIe6C83v8V0j/EkCSj7oFuuMRZ6IM99wOwc/Z5GmLbBeHgNV5ar4nExL7WT0kmYfOtaxB5y9fJGlFeA1DOj8omzQjB7aKUjpWyZWt8DFui4k+sZZFgPTTrHUdu1bP9zSR7t74uv5pmaMJOqccQ0aXxT/kuRDgIW13kOLOsA9djU8hAiy1szr25E6zwhxJ18cAEmoZTcwRYqIBCKxxAIwehLCG/96IhcaR1ifH8sykAp5NA1rzJg6eAKy+0C2M6l6I3/lRr+xtiTWolBzpeZ/pcRErFpffO2Adjg7+Rv2TI+Ye7WWSSxBIbk= ceph-9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:13:31.106 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbwJm9Pzi/ADG4K//ZEujUv+ALSdp7UwOjkR5+d8zfiXFE0KqNbTfaBZUJFU6Ga7kb3UHHWL560tmJlGnaxWPgSCa+RaJA0mx9r1DHVM/Whnz1Qtt+vGIbdjf255agDYxOTnPSlobJgq1/qcnpXQRA/kVKVZd9X5GA1cyxY+y8DE07raOiW3RCc5Y5XWQsQ9bFlk0xHsnbsIe6C83v8V0j/EkCSj7oFuuMRZ6IM99wOwc/Z5GmLbBeHgNV5ar4nExL7WT0kmYfOtaxB5y9fJGlFeA1DOj8omzQjB7aKUjpWyZWt8DFui4k+sZZFgPTTrHUdu1bP9zSR7t74uv5pmaMJOqccQ0aXxT/kuRDgIW13kOLOsA9djU8hAiy1szr25E6zwhxJ18cAEmoZTcwRYqIBCKxxAIwehLCG/96IhcaR1ifH8sykAp5NA1rzJg6eAKy+0C2M6l6I3/lRr+xtiTWolBzpeZ/pcRErFpffO2Adjg7+Rv2TI+Ye7WWSSxBIbk= ceph-9a9578d0-7435-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T14:13:31.226 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbwJm9Pzi/ADG4K//ZEujUv+ALSdp7UwOjkR5+d8zfiXFE0KqNbTfaBZUJFU6Ga7kb3UHHWL560tmJlGnaxWPgSCa+RaJA0mx9r1DHVM/Whnz1Qtt+vGIbdjf255agDYxOTnPSlobJgq1/qcnpXQRA/kVKVZd9X5GA1cyxY+y8DE07raOiW3RCc5Y5XWQsQ9bFlk0xHsnbsIe6C83v8V0j/EkCSj7oFuuMRZ6IM99wOwc/Z5GmLbBeHgNV5ar4nExL7WT0kmYfOtaxB5y9fJGlFeA1DOj8omzQjB7aKUjpWyZWt8DFui4k+sZZFgPTTrHUdu1bP9zSR7t74uv5pmaMJOqccQ0aXxT/kuRDgIW13kOLOsA9djU8hAiy1szr25E6zwhxJ18cAEmoZTcwRYqIBCKxxAIwehLCG/96IhcaR1ifH8sykAp5NA1rzJg6eAKy+0C2M6l6I3/lRr+xtiTWolBzpeZ/pcRErFpffO2Adjg7+Rv2TI+Ye7WWSSxBIbk= ceph-9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:13:31.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T14:13:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:13:31.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:31 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1400562895' entity='client.admin' 2024-09-16T14:13:34.697 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T14:13:34.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T14:13:35.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:35 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/555917079' entity='client.admin' 2024-09-16T14:13:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:13:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:13:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:36.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:36.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:37.793 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-09-16T14:13:37.793 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:13:37.794 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-09-16T14:13:37.813 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:13:37.813 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:13:37.871 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-09-16T14:13:37.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch host add smithi189 2024-09-16T14:13:38.073 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:13:38.397 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:13:38.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:38.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:38.398 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:39.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:39 smithi012 ceph-mon[25528]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:13:41.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:40 smithi012 ceph-mon[25528]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:43.004 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-09-16T14:13:43.004 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:42 smithi012 ceph-mon[25528]: Deploying cephadm binary to smithi189 2024-09-16T14:13:43.004 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:42 smithi012 ceph-mon[25528]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:43.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T14:13:43.771 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:13:44.218 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:44.218 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:44 smithi012 ceph-mon[25528]: Added host smithi189 2024-09-16T14:13:44.218 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:44.218 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:44.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:44.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:44 smithi012 ceph-mon[25528]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:13:45.427 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-09-16T14:13:45.427 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:45 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:46.029 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T14:13:46.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T14:13:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:13:46.639 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:46 smithi012 ceph-mon[25528]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:13:46.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:46 smithi012 ceph-mon[25528]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-09-16T14:13:48.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/775679914' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:48.823 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-09-16T14:13:48.823 INFO:tasks.cephadm:Adding mon.c on smithi012 2024-09-16T14:13:48.823 INFO:tasks.cephadm:Adding mon.b on smithi189 2024-09-16T14:13:48.823 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c;smithi189:172.21.15.189=b' 2024-09-16T14:13:49.020 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:13:49.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:49 smithi012 ceph-mon[25528]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:13:49.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:49 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/775679914' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T14:13:49.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:49 smithi012 ceph-mon[25528]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:13:50.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:50 smithi012 ceph-mon[25528]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:50.674 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-09-16T14:13:51.271 DEBUG:teuthology.orchestra.run.smithi012:mon.c> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T14:13:51.274 DEBUG:teuthology.orchestra.run.smithi189:mon.b> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T14:13:51.277 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T14:13:51.277 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T14:13:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c;smithi189:172.21.15.189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:13:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c;smithi189:172.21.15.189=b;count:3 2024-09-16T14:13:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:13:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:13:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:13:52.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:13:52.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:13:52.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:13:52.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: Deploying daemon mon.b on smithi189 2024-09-16T14:13:52.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:51 smithi012 ceph-mon[25528]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:53.193 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:13:54.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:53 smithi012 ceph-mon[25528]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:13:55.320 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:13:55.320 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"9a9578d0-7435-11ef-bceb-c7b262605968","modified":"2024-09-16T14:12:21.422304Z","created":"2024-09-16T14:12:21.422304Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T14:13:55.320 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-16T14:13:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:13:55 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/2421007156' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:13:56.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:13:55 smithi189 systemd[1]: Started Ceph mon.b for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:13:57.143 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T14:13:57.144 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T14:13:57.361 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:14:01.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:14:01.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: mon.a calling monitor election 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: mon.b calling monitor election 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:01.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: fsmap 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: mgrmap e13: y(active, since 39s) 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: overall HEALTH_OK 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:01.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:00 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:14:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: mon.a calling monitor election 2024-09-16T14:14:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: mon.b calling monitor election 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: fsmap 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: mgrmap e13: y(active, since 39s) 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: overall HEALTH_OK 2024-09-16T14:14:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:01.340 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:01.340 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:01.340 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:01.340 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:00 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:02.079 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:14:02.079 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"9a9578d0-7435-11ef-bceb-c7b262605968","modified":"2024-09-16T14:13:55.825450Z","created":"2024-09-16T14:12:21.422304Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T14:14:02.079 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-16T14:14:02.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:01 smithi012 ceph-mon[25528]: Deploying daemon mon.c on smithi012 2024-09-16T14:14:02.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:01 smithi012 ceph-mon[25528]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:02.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:01 smithi189 ceph-mon[30490]: Deploying daemon mon.c on smithi012 2024-09-16T14:14:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:01 smithi189 ceph-mon[30490]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:03.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:02 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/4047682044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:03.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:02 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/4047682044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:03.609 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T14:14:03.609 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T14:14:03.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:03 smithi012 systemd[1]: Starting Ceph mon.c for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:14:03.813 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:14:04.128 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:03 smithi012 ceph-mon[25528]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:03 smithi012 podman[38168]: 2024-09-16 14:14:03.76849071 +0000 UTC m=+0.016216232 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:03 smithi012 podman[38168]: 2024-09-16 14:14:03.872036932 +0000 UTC m=+0.119762454 container create b0ec5160f6fa8ffd6cd55ddce72465cfd45d24c0932fa6facccb7c8e36f8f798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 podman[38168]: 2024-09-16 14:14:04.074526893 +0000 UTC m=+0.322252416 container init b0ec5160f6fa8ffd6cd55ddce72465cfd45d24c0932fa6facccb7c8e36f8f798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 podman[38168]: 2024-09-16 14:14:04.07793069 +0000 UTC m=+0.325656208 container start b0ec5160f6fa8ffd6cd55ddce72465cfd45d24c0932fa6facccb7c8e36f8f798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 2 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pidfile_write: ignore empty --pid-file 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: load: jerasure load: lrc 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: RocksDB version: 7.9.2 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Git sha 0 2024-09-16T14:14:04.129 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: DB SUMMARY 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: DB Session ID: WZ6KT4YYPOU8MTI1HOLE 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: CURRENT file: CURRENT 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-16T14:14:04.130 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.error_if_exists: 0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.create_if_missing: 0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.paranoid_checks: 1 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.env: 0x559b88d60bc0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.info_log: 0x559b8a74cbe0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.statistics: (nil) 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.use_fsync: 0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_log_file_size: 0 2024-09-16T14:14:04.131 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.allow_fallocate: 1 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.use_direct_reads: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.db_log_dir: 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.wal_dir: 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T14:14:04.132 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.write_buffer_manager: 0x559b8a75a320 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T14:14:04.133 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.unordered_write: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.row_cache: None 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.wal_filter: None 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.two_write_queues: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.wal_compression: 0 2024-09-16T14:14:04.134 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.atomic_flush: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.log_readahead_size: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_background_jobs: 2 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_background_compactions: -1 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_subcompactions: 1 2024-09-16T14:14:04.135 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_open_files: -1 2024-09-16T14:14:04.136 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_background_flushes: -1 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Compression algorithms supported: 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kZSTD supported: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kXpressCompression supported: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kBZip2Compression supported: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kLZ4Compression supported: 1 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kZlibCompression supported: 1 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T14:14:04.138 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: kSnappyCompression supported: 1 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.merge_operator: 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_filter: None 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b8a74c7e0) 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T14:14:04.139 INFO:journalctl@ceph.mon.c.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: index_type: 0 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: data_block_index_type: 0 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: index_shortening: 1 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: checksum: 4 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: no_block_cache: 0 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_cache: 0x559b8a7411f0 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_cache_options: 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: capacity : 536870912 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: num_shard_bits : 4 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: strict_capacity_limit : 0 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-09-16T14:14:04.140 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_cache_compressed: (nil) 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: persistent_cache: (nil) 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_size: 4096 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_size_deviation: 10 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_restart_interval: 16 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: index_block_restart_interval: 1 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: metadata_block_size: 4096 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: partition_filters: 0 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: use_delta_encoding: 1 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: filter_policy: bloomfilter 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: whole_key_filtering: 1 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: verify_compression: 0 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: format_version: 5 2024-09-16T14:14:04.141 INFO:journalctl@ceph.mon.c.smithi012.stdout: enable_index_compression: 1 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout: block_align: 0 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout: max_auto_readahead_size: 262144 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout: prepopulate_block_cache: 0 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout: initial_auto_readahead_size: 8192 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression: NoCompression 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.num_levels: 7 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T14:14:04.142 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T14:14:04.143 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T14:14:04.144 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T14:14:04.145 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.inplace_update_support: 0 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T14:14:04.146 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.bloom_locality: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.max_successive_merges: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.ttl: 2592000 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.enable_blob_files: false 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.min_blob_size: 0 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T14:14:04.147 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4aa8e0d1-70cf-471e-987c-935f1c545ea4 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726496044104763, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726496044105154, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726496044, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4aa8e0d1-70cf-471e-987c-935f1c545ea4", "db_session_id": "WZ6KT4YYPOU8MTI1HOLE", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726496044105251, "job": 1, "event": "recovery_finished"} 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T14:14:04.148 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559b8a76ce00 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: DB pointer 0x559b8a860000 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: ** DB Stats ** 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: 2024-09-16T14:14:04.149 INFO:journalctl@ceph.mon.c.smithi012.stdout: ** Compaction Stats [default] ** 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: ** Compaction Stats [default] ** 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: 2024-09-16T14:14:04.150 INFO:journalctl@ceph.mon.c.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: Block cache BinnedLRUCache@0x559b8a7411f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: using public_addrv [v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0] 2024-09-16T14:14:04.151 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: starting mon.c rank -1 at public addrs [v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0] at bind addrs [v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(???) e0 preinit fsid 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout: e1 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout: legacy client fscid: -1 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout: 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout: No filesystems configured 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T14:14:04.152 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/775679914' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c;smithi189:172.21.15.189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c;smithi189:172.21.15.189=b;count:3 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:04.153 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: Deploying daemon mon.b on smithi189 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/2421007156' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.a calling monitor election 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.b calling monitor election 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: monmap e2: 2 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: fsmap 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mgrmap e13: y(active, since 39s) 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: overall HEALTH_OK 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:04.155 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:04.156 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: Deploying daemon mon.c on smithi012 2024-09-16T14:14:04.156 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.156 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:04.156 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/4047682044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:04.156 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.156 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 ceph-mon[38187]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T14:14:04.222 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:03 smithi189 ceph-mon[30490]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:04.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 bash[38168]: b0ec5160f6fa8ffd6cd55ddce72465cfd45d24c0932fa6facccb7c8e36f8f798 2024-09-16T14:14:04.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:04 smithi012 systemd[1]: Started Ceph mon.c for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: mon.a calling monitor election 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: mon.b calling monitor election 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: mon.c calling monitor election 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0],c=[v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: fsmap 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: mgrmap e13: y(active, since 47s) 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: overall HEALTH_OK 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: mon.a calling monitor election 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: mon.b calling monitor election 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:09.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: mon.c calling monitor election 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0],c=[v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: fsmap 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: mgrmap e13: y(active, since 47s) 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: overall HEALTH_OK 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.531 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.532 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: mon.a calling monitor election 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: mon.b calling monitor election 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: mon.c calling monitor election 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:09.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: monmap e3: 3 mons at {a=[v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0],b=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0],c=[v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: fsmap 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: mgrmap e13: y(active, since 47s) 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: overall HEALTH_OK 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-16T14:14:09.933 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"9a9578d0-7435-11ef-bceb-c7b262605968","modified":"2024-09-16T14:14:04.134326Z","created":"2024-09-16T14:12:21.422304Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3301","nonce":0},{"type":"v1","addr":"172.21.15.12:6790","nonce":0}]},"addr":"172.21.15.12:6790/0","public_addr":"172.21.15.12:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-16T14:14:09.933 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-09-16T14:14:10.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:10.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:10 smithi189 ceph-mon[30490]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:10.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:10 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/514801211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:10.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:10.517 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:10.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[38187]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/514801211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[25528]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/514801211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T14:14:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:14:10.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:10.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:10.858 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T14:14:10.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T14:14:11.059 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:14:11.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.474 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:14:11.476 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:11.477 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:11.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:11.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:11.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:12.976 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[25528]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[25528]: Reconfiguring daemon mon.a on smithi012 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[25528]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[38187]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[38187]: Reconfiguring daemon mon.a on smithi012 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[38187]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:12.977 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:12 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:13.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:12 smithi189 ceph-mon[30490]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T14:14:13.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:12 smithi189 ceph-mon[30490]: Reconfiguring daemon mon.a on smithi012 2024-09-16T14:14:13.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:12 smithi189 ceph-mon[30490]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:13.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:12 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:13.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:12 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:13.917 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:13.917 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[38187]: Reconfiguring mon.c (monmap changed)... 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[38187]: Reconfiguring daemon mon.c on smithi012 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[25528]: Reconfiguring mon.c (monmap changed)... 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:13.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:13 smithi012 ceph-mon[25528]: Reconfiguring daemon mon.c on smithi012 2024-09-16T14:14:14.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:13 smithi189 ceph-mon[30490]: Reconfiguring mon.c (monmap changed)... 2024-09-16T14:14:14.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:14.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:14.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:14.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:13 smithi189 ceph-mon[30490]: Reconfiguring daemon mon.c on smithi012 2024-09-16T14:14:14.196 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:14:14.196 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-09-16T14:14:14.196 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:14:14.196 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] [v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0] 2024-09-16T14:14:14.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:14.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1023950330' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:14.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1023950330' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:14.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:14.930 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:14.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1023950330' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:14.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:14.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T14:14:14.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T14:14:14.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:15.106 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T14:14:15.107 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:14:15.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T14:14:15.136 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:14:15.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:14:15.203 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:14:15.203 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T14:14:15.230 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:14:15.231 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:14:15.295 INFO:tasks.cephadm:Adding mgr.y on smithi012 2024-09-16T14:14:15.295 INFO:tasks.cephadm:Adding mgr.x on smithi189 2024-09-16T14:14:15.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi012=y;smithi189=x' 2024-09-16T14:14:15.852 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:14:15.967 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:15 smithi189 ceph-mon[30490]: Reconfiguring mon.b (monmap changed)... 2024-09-16T14:14:15.967 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:15 smithi189 ceph-mon[30490]: Reconfiguring daemon mon.b on smithi189 2024-09-16T14:14:16.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:15 smithi012 ceph-mon[25528]: Reconfiguring mon.b (monmap changed)... 2024-09-16T14:14:16.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:15 smithi012 ceph-mon[25528]: Reconfiguring daemon mon.b on smithi189 2024-09-16T14:14:16.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:15 smithi012 ceph-mon[38187]: Reconfiguring mon.b (monmap changed)... 2024-09-16T14:14:16.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:15 smithi012 ceph-mon[38187]: Reconfiguring daemon mon.b on smithi189 2024-09-16T14:14:16.964 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:16.964 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:16.964 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:16.965 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:16.965 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:16.965 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:16.965 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:17.555 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-09-16T14:14:18.098 DEBUG:teuthology.orchestra.run.smithi189:mgr.x> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T14:14:18.100 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T14:14:18.101 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:14:18.101 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:14:18.121 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T14:14:18.121 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/nvme4n1 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/nvme4n1 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:13:33.983577006 +0000 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:42.841524377 +0000 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:42.841524377 +0000 2024-09-16T14:14:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:42.837524413 +0000 2024-09-16T14:14:18.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-16T14:14:18.251 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:14:18.251 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:14:18.251 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000253328 s, 2.0 MB/s 2024-09-16T14:14:18.252 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-16T14:14:18.312 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/nvme3n1 2024-09-16T14:14:18.370 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/nvme3n1 2024-09-16T14:14:18.370 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:18.370 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-16T14:14:18.370 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:18.371 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:18.371 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:13:33.972577099 +0000 2024-09-16T14:14:18.371 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:42.692525680 +0000 2024-09-16T14:14:18.371 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:42.692525680 +0000 2024-09-16T14:14:18.371 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:42.687525724 +0000 2024-09-16T14:14:18.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-16T14:14:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:14:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:14:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000219635 s, 2.3 MB/s 2024-09-16T14:14:18.439 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-16T14:14:18.496 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/nvme2n1 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/nvme2n1 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:13:33.961577191 +0000 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:42.560526834 +0000 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:42.560526834 +0000 2024-09-16T14:14:18.554 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:42.556526869 +0000 2024-09-16T14:14:18.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-16T14:14:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:14:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:14:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000225292 s, 2.3 MB/s 2024-09-16T14:14:18.621 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-16T14:14:18.679 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/nvme1n1 2024-09-16T14:14:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/nvme1n1 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 14:13:33.951577276 +0000 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 14:11:42.412528128 +0000 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 14:11:42.412528128 +0000 2024-09-16T14:14:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-16 14:11:42.407528172 +0000 2024-09-16T14:14:18.737 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-16T14:14:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T14:14:18.800 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T14:14:18.801 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000298495 s, 1.7 MB/s 2024-09-16T14:14:18.802 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi012=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: Saving service mgr spec with placement smithi012=y;smithi189=x;count:2 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:18.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:18.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T14:14:18.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T14:14:18.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T14:14:18.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:18.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: Deploying daemon mgr.x on smithi189 2024-09-16T14:14:18.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:18 smithi189 ceph-mon[30490]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:18.862 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:14:18.863 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:14:18.879 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T14:14:18.879 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme4n1 2024-09-16T14:14:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme4n1 2024-09-16T14:14:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:18.936 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-16T14:14:18.936 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:18.936 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:18.936 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:13:46.174971540 +0000 2024-09-16T14:14:18.937 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:44.456365450 +0000 2024-09-16T14:14:18.937 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:44.456365450 +0000 2024-09-16T14:14:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:44.451365549 +0000 2024-09-16T14:14:18.937 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-16T14:14:19.000 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:14:19.000 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:14:19.000 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000195843 s, 2.6 MB/s 2024-09-16T14:14:19.001 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-16T14:14:19.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi012=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:19.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: Saving service mgr spec with placement smithi012=y;smithi189=x;count:2 2024-09-16T14:14:19.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: Deploying daemon mgr.x on smithi189 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[25528]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi012=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: Saving service mgr spec with placement smithi012=y;smithi189=x;count:2 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:19.029 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: Deploying daemon mgr.x on smithi189 2024-09-16T14:14:19.030 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:18 smithi012 ceph-mon[38187]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:19.057 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme3n1 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme3n1 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:13:46.164971737 +0000 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:44.313368273 +0000 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:44.313368273 +0000 2024-09-16T14:14:19.113 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:44.308368372 +0000 2024-09-16T14:14:19.114 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-16T14:14:19.177 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:14:19.177 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:14:19.177 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000262442 s, 2.0 MB/s 2024-09-16T14:14:19.178 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-16T14:14:19.236 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme2n1 2024-09-16T14:14:19.299 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme2n1 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:13:46.153971953 +0000 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:44.174371017 +0000 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:44.174371017 +0000 2024-09-16T14:14:19.300 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:44.170371096 +0000 2024-09-16T14:14:19.300 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-16T14:14:19.462 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:14:19.463 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:14:19.463 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000229673 s, 2.2 MB/s 2024-09-16T14:14:19.464 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-16T14:14:19.480 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/nvme1n1 2024-09-16T14:14:19.543 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/nvme1n1 2024-09-16T14:14:19.543 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T14:14:19.543 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-16T14:14:19.544 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T14:14:19.544 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T14:14:19.544 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-16 14:13:46.142972169 +0000 2024-09-16T14:14:19.544 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-16 14:11:44.017374116 +0000 2024-09-16T14:14:19.544 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-16 14:11:44.017374116 +0000 2024-09-16T14:14:19.544 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-16 14:11:44.011374235 +0000 2024-09-16T14:14:19.544 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-16T14:14:19.664 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-16T14:14:19.664 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-16T14:14:19.664 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000237247 s, 2.2 MB/s 2024-09-16T14:14:19.665 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-16T14:14:19.681 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/nvme1n1... 2024-09-16T14:14:19.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-16T14:14:19.723 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:19 smithi189 systemd[1]: Starting Ceph mgr.x for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:14:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:14:20.000 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:19 smithi189 ceph-mon[30490]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:20.000 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:19 smithi189 podman[32333]: 2024-09-16 14:14:19.89381514 +0000 UTC m=+0.111845140 container create 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T14:14:20.001 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:19 smithi189 podman[32333]: 2024-09-16 14:14:19.79716889 +0000 UTC m=+0.015198845 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:14:20.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:19 smithi012 ceph-mon[25528]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:20.013 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:19 smithi012 ceph-mon[38187]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:20.280 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 podman[32333]: 2024-09-16 14:14:20.095982512 +0000 UTC m=+0.314012468 container init 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T14:14:20.280 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 podman[32333]: 2024-09-16 14:14:20.098869796 +0000 UTC m=+0.316899759 container start 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T14:14:20.280 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-mgr[32352]: -- 172.21.15.189:0/1208472343 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55feae6e6d00 con 0x55feae6bb000 2024-09-16T14:14:20.280 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 bash[32333]: 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 2024-09-16T14:14:20.281 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 systemd[1]: Started Ceph mgr.x for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:14:20.281 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:20.217+0000 7f6e6f23a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:14:20.281 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:20.278+0000 7f6e6f23a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:14:20.588 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:20.435+0000 7f6e6f23a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:14:20.588 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:20.503+0000 7f6e6f23a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:14:20.976 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:20.670+0000 7f6e6f23a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:14:20.976 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:20.812+0000 7f6e6f23a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:14:21.258 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:21.137+0000 7f6e6f23a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:14:21.258 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:21.196+0000 7f6e6f23a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:14:21.518 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:21.399+0000 7f6e6f23a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:14:21.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/nvme1n1 2024-09-16T14:14:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:14:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-16T14:14:21.587 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-09-16T14:14:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-09-16T14:14:21.587 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014065 s, 746 MB/s 2024-09-16T14:14:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T14:14:22.508 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: Reconfiguring daemon mgr.y on smithi012 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T14:14:22.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T14:14:22.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:22.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: Reconfiguring daemon mgr.y on smithi012 2024-09-16T14:14:22.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:22.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.588 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:22.310+0000 7f6e6f23a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:14:22.588 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:22.378+0000 7f6e6f23a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:14:22.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: Reconfiguring daemon mgr.y on smithi012 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:22.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:23.259 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi012:/dev/nvme1n1 2024-09-16T14:14:23.363 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.109+0000 7f6e6f23a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:14:23.363 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.223+0000 7f6e6f23a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:14:23.363 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.361+0000 7f6e6f23a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:14:23.459 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:14:23.634 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.422+0000 7f6e6f23a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:14:23.634 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.565+0000 7f6e6f23a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:14:23.634 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.632+0000 7f6e6f23a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:14:23.915 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:23.915 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:23.915 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-mon[30490]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.691+0000 7f6e6f23a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.847+0000 7f6e6f23a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:14:23.916 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.913+0000 7f6e6f23a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:14:24.181 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:23.979+0000 7f6e6f23a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:14:24.181 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:24.061+0000 7f6e6f23a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:14:24.181 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:24.120+0000 7f6e6f23a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:14:24.181 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:24.179+0000 7f6e6f23a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:14:24.233 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:24.233 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:24.233 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:24.233 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[38187]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:24.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:23 smithi012 ceph-mon[25528]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:25.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[25528]: Standby manager daemon x started 2024-09-16T14:14:25.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T14:14:25.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T14:14:25.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T14:14:25.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T14:14:25.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[38187]: Standby manager daemon x started 2024-09-16T14:14:25.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T14:14:25.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T14:14:25.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T14:14:25.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:25 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T14:14:25.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:25 smithi189 ceph-mon[30490]: Standby manager daemon x started 2024-09-16T14:14:25.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:25 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T14:14:25.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:25 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T14:14:25.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:25 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T14:14:25.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:25 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/683490579' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T14:14:25.339 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:14:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:14:24.953+0000 7f6e6f23a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:14:26.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-16T14:14:26.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T14:14:26.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:26.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[25528]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:14:26.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:26.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:26 smithi012 ceph-mon[38187]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:26.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-16T14:14:26.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T14:14:26.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:26.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:14:26.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:14:26.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:26.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:26 smithi189 ceph-mon[30490]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:27.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/54916376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc317e5-4d35-45ec-9771-6cbee7431be3"}]: dispatch 2024-09-16T14:14:27.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/54916376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fc317e5-4d35-45ec-9771-6cbee7431be3"}]': finished 2024-09-16T14:14:27.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[25528]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T14:14:27.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:27.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/54916376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc317e5-4d35-45ec-9771-6cbee7431be3"}]: dispatch 2024-09-16T14:14:27.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/54916376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fc317e5-4d35-45ec-9771-6cbee7431be3"}]': finished 2024-09-16T14:14:27.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[38187]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T14:14:27.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:27 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:27 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/54916376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fc317e5-4d35-45ec-9771-6cbee7431be3"}]: dispatch 2024-09-16T14:14:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:27 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/54916376' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fc317e5-4d35-45ec-9771-6cbee7431be3"}]': finished 2024-09-16T14:14:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:27 smithi189 ceph-mon[30490]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T14:14:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:27 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:28.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:28 smithi012 ceph-mon[25528]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:28.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:28 smithi012 ceph-mon[38187]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:28.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:28 smithi189 ceph-mon[30490]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:29.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:29 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3972832774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:14:29.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:29 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3972832774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:14:29.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:29 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3972832774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:14:30.733 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:30 smithi012 ceph-mon[25528]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:30.734 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:30 smithi012 ceph-mon[38187]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:30.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:30 smithi189 ceph-mon[30490]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:32.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:31 smithi012 ceph-mon[25528]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:32.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:31 smithi012 ceph-mon[38187]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:32.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:31 smithi189 ceph-mon[30490]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:34.003 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:33 smithi012 ceph-mon[38187]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:34.004 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:33 smithi012 ceph-mon[25528]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:33 smithi189 ceph-mon[30490]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T14:14:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:34 smithi012 ceph-mon[25528]: Deploying daemon osd.0 on smithi012 2024-09-16T14:14:35.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T14:14:35.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:34 smithi012 ceph-mon[38187]: Deploying daemon osd.0 on smithi012 2024-09-16T14:14:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T14:14:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:35.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:34 smithi189 ceph-mon[30490]: Deploying daemon osd.0 on smithi012 2024-09-16T14:14:35.889 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:35 smithi012 ceph-mon[38187]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:35.890 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:35 smithi012 ceph-mon[25528]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:35 smithi189 ceph-mon[30490]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:37.858 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:37 smithi012 ceph-mon[25528]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:37.858 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:37 smithi012 ceph-mon[38187]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:38.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:37 smithi189 ceph-mon[30490]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:39.525 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:39 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:39.525 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:39 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:39.525 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:39 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:39.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:39 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:39.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:39 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:39.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:39 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:39.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:39 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:39.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:39 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:39.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:39 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:40.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:40 smithi012 ceph-mon[38187]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:40.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:40 smithi012 ceph-mon[25528]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:40.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:40 smithi189 ceph-mon[30490]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:42.393 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T14:14:42.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:42.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:42 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:42.966 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T14:14:42.968 INFO:tasks.cephadm:Deploying osd.1 on smithi012 with /dev/nvme2n1... 2024-09-16T14:14:42.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:43.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T14:14:43.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:14:43.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:43.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:42 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:44.055 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:14:44.085 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[38187]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T14:14:44.085 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[38187]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T14:14:44.085 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:44.085 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[38187]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:14:44.085 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[38187]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:44.085 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[25528]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T14:14:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[25528]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T14:14:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[25528]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:14:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:43 smithi012 ceph-mon[25528]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:44.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:43 smithi189 ceph-mon[30490]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T14:14:44.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:43 smithi189 ceph-mon[30490]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T14:14:44.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:43 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:44.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:43 smithi189 ceph-mon[30490]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:14:44.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:43 smithi189 ceph-mon[30490]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:44.685 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0[42022]: 2024-09-16T14:14:44.398+0000 7fa7205ad640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T14:14:44.685 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0[42022]: 2024-09-16T14:14:44.401+0000 7fa7183ce640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:14:45.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[38187]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:14:45.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[38187]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T14:14:45.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:45.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:45.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[38187]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' 2024-09-16T14:14:45.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[25528]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:14:45.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[25528]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T14:14:45.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:45.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:45.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:44 smithi012 ceph-mon[25528]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' 2024-09-16T14:14:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:44 smithi189 ceph-mon[30490]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:14:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:44 smithi189 ceph-mon[30490]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T14:14:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:44 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:44 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:44 smithi189 ceph-mon[30490]: from='osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591]' entity='osd.0' 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591] boot 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[25528]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:14:46.266 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:14:46.267 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:46.267 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591] boot 2024-09-16T14:14:46.267 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T14:14:46.267 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:46.267 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:45 smithi012 ceph-mon[38187]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: osd.0 [v2:172.21.15.12:6802/1318340591,v1:172.21.15.12:6803/1318340591] boot 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:14:46.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:45 smithi189 ceph-mon[30490]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T14:14:46.774 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/nvme2n1 2024-09-16T14:14:46.774 INFO:teuthology.orchestra.run.smithi012.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:14:46.779 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-16T14:14:46.795 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-09-16T14:14:46.795 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-09-16T14:14:46.796 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143979 s, 728 MB/s 2024-09-16T14:14:46.796 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-09-16T14:14:47.358 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi012 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi012 to 16068M 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[25528]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T14:14:47.359 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi012 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi012 to 16068M 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.360 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:47 smithi012 ceph-mon[38187]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T14:14:47.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi012:/dev/nvme2n1 2024-09-16T14:14:47.579 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:14:47.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi012 2024-09-16T14:14:47.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:14:47.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi012 to 16068M 2024-09-16T14:14:47.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:47.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:14:47.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:14:47.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:47 smithi189 ceph-mon[30490]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T14:14:48.623 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:48 smithi012 ceph-mon[38187]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:48.624 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:48 smithi012 ceph-mon[25528]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:48.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:48 smithi189 ceph-mon[30490]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:49.614 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:49 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:14:49.615 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:49 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:14:49.615 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:49 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:49.615 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:49 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:14:49.615 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:49 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:14:49.615 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:49 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:49.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:49 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:14:49.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:49 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:14:49.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:49 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:50.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:50 smithi012 ceph-mon[25528]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:50.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:50 smithi012 ceph-mon[25528]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:50 smithi012 ceph-mon[38187]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:50 smithi012 ceph-mon[38187]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:50.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:50 smithi189 ceph-mon[30490]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:14:50.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:50 smithi189 ceph-mon[30490]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1750994199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5dfe945-9adc-4b48-aad6-adb6427ebe6f"}]: dispatch 2024-09-16T14:14:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1750994199' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5dfe945-9adc-4b48-aad6-adb6427ebe6f"}]': finished 2024-09-16T14:14:52.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[25528]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T14:14:52.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:14:52.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1750994199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5dfe945-9adc-4b48-aad6-adb6427ebe6f"}]: dispatch 2024-09-16T14:14:52.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1750994199' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5dfe945-9adc-4b48-aad6-adb6427ebe6f"}]': finished 2024-09-16T14:14:52.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[38187]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T14:14:52.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:14:52.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:51 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1750994199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5dfe945-9adc-4b48-aad6-adb6427ebe6f"}]: dispatch 2024-09-16T14:14:52.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:51 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1750994199' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5dfe945-9adc-4b48-aad6-adb6427ebe6f"}]': finished 2024-09-16T14:14:52.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:51 smithi189 ceph-mon[30490]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T14:14:52.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:14:52.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:52 smithi012 ceph-mon[25528]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:52.747 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:52 smithi012 ceph-mon[38187]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:53.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:52 smithi189 ceph-mon[30490]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:54.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:53 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3658055964' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:14:54.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:53 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3658055964' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:14:54.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:53 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3658055964' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:14:54.891 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:54 smithi012 ceph-mon[25528]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:54.892 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:54 smithi012 ceph-mon[38187]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:55.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:54 smithi189 ceph-mon[30490]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:56.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:55 smithi012 ceph-mon[25528]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:56.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:55 smithi012 ceph-mon[38187]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:56.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:55 smithi189 ceph-mon[30490]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:58.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:57 smithi012 ceph-mon[25528]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:58.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:57 smithi012 ceph-mon[38187]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:58.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:57 smithi189 ceph-mon[30490]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T14:14:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:58 smithi012 ceph-mon[38187]: Deploying daemon osd.1 on smithi012 2024-09-16T14:14:58.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T14:14:58.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:58.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:58 smithi012 ceph-mon[25528]: Deploying daemon osd.1 on smithi012 2024-09-16T14:14:59.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T14:14:59.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:14:59.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:58 smithi189 ceph-mon[30490]: Deploying daemon osd.1 on smithi012 2024-09-16T14:14:59.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:14:59 smithi012 ceph-mon[38187]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:14:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:14:59 smithi012 ceph-mon[25528]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:00.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:14:59 smithi189 ceph-mon[30490]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:02.012 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:01 smithi012 ceph-mon[25528]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:02.012 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:01 smithi012 ceph-mon[38187]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:02.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:01 smithi189 ceph-mon[30490]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:03.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:03 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:03.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:03 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:03.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:03 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:03.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:03 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:03.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:03 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:03.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:03 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:03.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:03 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:03.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:03 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:03.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:03 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:04.538 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:04 smithi012 ceph-mon[38187]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:04.539 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:04 smithi012 ceph-mon[25528]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:04.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:04 smithi189 ceph-mon[30490]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:05.819 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:05 smithi012 ceph-mon[38187]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:05.819 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:05 smithi012 ceph-mon[25528]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:06.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:05 smithi189 ceph-mon[30490]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:06.623 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 1 on host 'smithi012' 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.969 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:06.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:06 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T14:15:07.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T14:15:07.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:07.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:07.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:06 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:07.299 DEBUG:teuthology.orchestra.run.smithi012:osd.1> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T14:15:07.301 INFO:tasks.cephadm:Deploying osd.2 on smithi012 with /dev/nvme3n1... 2024-09-16T14:15:07.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-16T14:15:08.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[38187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T14:15:08.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[38187]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T14:15:08.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[38187]: from='osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[38187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[38187]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[25528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[25528]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[25528]: from='osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[25528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:08.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:07 smithi012 ceph-mon[25528]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:08.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:07 smithi189 ceph-mon[30490]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T14:15:08.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:07 smithi189 ceph-mon[30490]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T14:15:08.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:07 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:07 smithi189 ceph-mon[30490]: from='osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:08.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:07 smithi189 ceph-mon[30490]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:08.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:07 smithi189 ceph-mon[30490]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:15:08.527 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1[49815]: 2024-09-16T14:15:08.303+0000 7f77742d8640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T14:15:08.527 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1[49815]: 2024-09-16T14:15:08.307+0000 7f776b8e2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[38187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[38187]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[38187]: from='osd.1 ' entity='osd.1' 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[25528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[25528]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:08.966 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:08 smithi012 ceph-mon[25528]: from='osd.1 ' entity='osd.1' 2024-09-16T14:15:09.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:08 smithi189 ceph-mon[30490]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:09.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:08 smithi189 ceph-mon[30490]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T14:15:09.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:09.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:09.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:08 smithi189 ceph-mon[30490]: from='osd.1 ' entity='osd.1' 2024-09-16T14:15:09.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:15:09.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740] boot 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[25528]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740] boot 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:09.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:09 smithi012 ceph-mon[38187]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: osd.1 [v2:172.21.15.12:6810/4046669740,v1:172.21.15.12:6811/4046669740] boot 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:15:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:09 smithi189 ceph-mon[30490]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T14:15:10.338 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/nvme3n1 2024-09-16T14:15:10.338 INFO:teuthology.orchestra.run.smithi012.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:15:10.343 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-16T14:15:10.362 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-09-16T14:15:10.363 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-09-16T14:15:10.363 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0177051 s, 592 MB/s 2024-09-16T14:15:10.363 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:11.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:11.711 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi012:/dev/nvme3n1 2024-09-16T14:15:11.914 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:15:12.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:12 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi012 2024-09-16T14:15:12.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:12 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi012 to 8034M 2024-09-16T14:15:12.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:12 smithi189 ceph-mon[30490]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:12.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:12 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi012 2024-09-16T14:15:12.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:12 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi012 to 8034M 2024-09-16T14:15:12.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:12 smithi012 ceph-mon[25528]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:12.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:12 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi012 2024-09-16T14:15:12.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:12 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi012 to 8034M 2024-09-16T14:15:12.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:12 smithi012 ceph-mon[38187]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:13.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:15:13.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:15:13.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:13 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:13.934 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:15:13.934 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:15:13.935 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:13 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:14.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:15:14.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:15:14.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:13 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:14.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:14 smithi012 ceph-mon[25528]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:15:14.931 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:14 smithi012 ceph-mon[25528]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:14.931 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:14 smithi012 ceph-mon[38187]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:15:14.931 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:14 smithi012 ceph-mon[38187]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:15.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:14 smithi189 ceph-mon[30490]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:15:15.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:14 smithi189 ceph-mon[30490]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:15.853 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:15 smithi012 ceph-mon[25528]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:15.854 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:15 smithi012 ceph-mon[38187]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:16.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:15 smithi189 ceph-mon[30490]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2610181739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]: dispatch 2024-09-16T14:15:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]: dispatch 2024-09-16T14:15:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]': finished 2024-09-16T14:15:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[25528]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T14:15:17.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2610181739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]: dispatch 2024-09-16T14:15:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]: dispatch 2024-09-16T14:15:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]': finished 2024-09-16T14:15:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[38187]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T14:15:17.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:16 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:17.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:16 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2610181739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]: dispatch 2024-09-16T14:15:17.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:16 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]: dispatch 2024-09-16T14:15:17.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:16 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37dca01c-8cbf-44df-ac06-f83d9ab1143f"}]': finished 2024-09-16T14:15:17.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:16 smithi189 ceph-mon[30490]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T14:15:17.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:16 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:18.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:17 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1434346017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:15:18.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:17 smithi012 ceph-mon[25528]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:17 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1434346017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:15:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:17 smithi012 ceph-mon[38187]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:17 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1434346017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:15:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:17 smithi189 ceph-mon[30490]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:19.875 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:19 smithi012 ceph-mon[25528]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:19.875 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:19 smithi012 ceph-mon[38187]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:20.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:19 smithi189 ceph-mon[30490]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:21.886 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:21 smithi012 ceph-mon[38187]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:21.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:21 smithi012 ceph-mon[25528]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:21 smithi189 ceph-mon[30490]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:23.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T14:15:23.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:23.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:22 smithi012 ceph-mon[25528]: Deploying daemon osd.2 on smithi012 2024-09-16T14:15:23.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T14:15:23.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:23.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:22 smithi012 ceph-mon[38187]: Deploying daemon osd.2 on smithi012 2024-09-16T14:15:23.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T14:15:23.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:23.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:22 smithi189 ceph-mon[30490]: Deploying daemon osd.2 on smithi012 2024-09-16T14:15:24.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:23 smithi012 ceph-mon[25528]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:23 smithi012 ceph-mon[38187]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:23 smithi189 ceph-mon[30490]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:25.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:25 smithi012 ceph-mon[25528]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:25.917 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:25 smithi012 ceph-mon[38187]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:26.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:25 smithi189 ceph-mon[30490]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:27.750 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:27 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:27.750 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:27 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:27.750 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:27 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:27.751 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:27 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:27.751 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:27 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:27.751 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:27 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:27 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:27 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:27.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:27 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:28.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:28 smithi012 ceph-mon[25528]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:28.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:28 smithi012 ceph-mon[38187]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:28.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:28 smithi189 ceph-mon[30490]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:29.835 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:29 smithi012 ceph-mon[38187]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:29.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:29 smithi012 ceph-mon[25528]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:30.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:29 smithi189 ceph-mon[30490]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:30.884 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 2 on host 'smithi012' 2024-09-16T14:15:31.184 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.184 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.184 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:31.184 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:31.184 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T14:15:31.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T14:15:31.186 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:31.186 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.186 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:31 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:31.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:31 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:31.520 DEBUG:teuthology.orchestra.run.smithi012:osd.2> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T14:15:31.522 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/nvme4n1... 2024-09-16T14:15:31.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-16T14:15:32.214 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:15:32.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[25528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T14:15:32.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[25528]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T14:15:32.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:32.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[25528]: from='osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[25528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[25528]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[38187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[38187]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[38187]: from='osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[38187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:32.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-mon[38187]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:32.529 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2[57114]: 2024-09-16T14:15:32.103+0000 7ffa89e6b640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T14:15:32.529 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:32 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2[57114]: 2024-09-16T14:15:32.106+0000 7ffa85c94640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:15:32.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:32 smithi189 ceph-mon[30490]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T14:15:32.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:32 smithi189 ceph-mon[30490]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T14:15:32.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:32 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:32.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:32 smithi189 ceph-mon[30490]: from='osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:32.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:32 smithi189 ceph-mon[30490]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:32.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:32 smithi189 ceph-mon[30490]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[25528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[25528]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[38187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[38187]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T14:15:33.478 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:33.479 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:33 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:33.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:33 smithi189 ceph-mon[30490]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:33.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:33 smithi189 ceph-mon[30490]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T14:15:33.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:33 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:33.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:33 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[38187]: osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517] boot 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[38187]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:15:34.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[25528]: osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517] boot 2024-09-16T14:15:34.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[25528]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T14:15:34.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:34.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:34 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:15:34.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:34 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:15:34.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:34 smithi189 ceph-mon[30490]: osd.2 [v2:172.21.15.12:6818/759885517,v1:172.21.15.12:6819/759885517] boot 2024-09-16T14:15:34.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:34 smithi189 ceph-mon[30490]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T14:15:34.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:15:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/nvme4n1 2024-09-16T14:15:34.907 INFO:teuthology.orchestra.run.smithi012.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:15:34.911 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-16T14:15:34.928 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-09-16T14:15:34.928 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-09-16T14:15:34.928 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145269 s, 722 MB/s 2024-09-16T14:15:34.928 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-09-16T14:15:35.159 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[38187]: pgmap v74: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:35.159 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T14:15:35.159 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T14:15:35.159 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[38187]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T14:15:35.159 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T14:15:35.159 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[25528]: pgmap v74: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:35.160 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T14:15:35.160 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T14:15:35.160 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[25528]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T14:15:35.160 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:35 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T14:15:35.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:35 smithi189 ceph-mon[30490]: pgmap v74: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:35.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:35 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T14:15:35.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:35 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T14:15:35.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:35 smithi189 ceph-mon[30490]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T14:15:35.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:35 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T14:15:35.852 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:35 smithi012 sudo[62722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-16T14:15:35.852 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:35 smithi012 sudo[62722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:35.852 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:35 smithi012 sudo[62722]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:35.852 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:35 smithi012 sudo[62722]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:35.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi012:/dev/nvme4n1 2024-09-16T14:15:36.412 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:15:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T14:15:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:36 smithi012 ceph-mon[25528]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T14:15:36.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:36 smithi012 ceph-mon[25528]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:36.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-16T14:15:36.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:36.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62759]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:36.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62759]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:36.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:36 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T14:15:36.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:36 smithi012 ceph-mon[38187]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T14:15:36.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:36 smithi012 ceph-mon[38187]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:36.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:36 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T14:15:36.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:36 smithi189 ceph-mon[30490]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T14:15:36.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:36 smithi189 ceph-mon[30490]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:36.875 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-16T14:15:36.875 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:36.875 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62836]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:36.876 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:36 smithi012 sudo[62836]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:37.127 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62856]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-16T14:15:37.127 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:37.127 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62856]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:37.127 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62856]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi012 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.509 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi012 to 5356M 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.510 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi012 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi012 to 5356M 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:37 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T14:15:37.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T14:15:37.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi012 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi012 to 5356M 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:37 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:37.775 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62860]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-16T14:15:37.776 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:37.776 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62860]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:37.776 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62860]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:38.027 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62892]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-16T14:15:38.027 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62892]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:38.027 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62892]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:38.027 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:15:37 smithi012 sudo[62892]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:38.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T14:15:38.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:38.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:38.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62920]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:39.249 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:38 smithi189 ceph-mon[30490]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:39.249 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:39.249 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:38 smithi189 ceph-mon[30490]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:38 smithi012 ceph-mon[38187]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:38 smithi012 ceph-mon[38187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62924]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:39.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 sudo[62924]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:39.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 ceph-mon[25528]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:39.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:39.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:38 smithi012 ceph-mon[25528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:39.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32994]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T14:15:39.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32994]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:39.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32994]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:39.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32994]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T14:15:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32997]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:39 smithi189 sudo[32997]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:39 smithi012 sudo[63066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T14:15:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:39 smithi012 sudo[63066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:39 smithi012 sudo[63066]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:39 smithi012 sudo[63066]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 sudo[63100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 sudo[63100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 sudo[63100]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 sudo[63100]: pam_unix(sudo:session): session closed for user root 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.643 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.644 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[38187]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:40.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.646 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:40 smithi012 ceph-mon[25528]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:40.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:15:40.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:40.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:40.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:15:40.840 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:15:40.840 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:15:40.840 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:40 smithi189 ceph-mon[30490]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:41.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:41.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:41.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2619119200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]: dispatch 2024-09-16T14:15:41.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]: dispatch 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]': finished 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2619119200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]: dispatch 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]: dispatch 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]': finished 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T14:15:41.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:41 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2619119200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]: dispatch 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]: dispatch 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98f70795-ebfd-4724-b972-843fdbfe69bc"}]': finished 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T14:15:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:41 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:42.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:42 smithi012 ceph-mon[25528]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:42.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:42 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2406599162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:15:42.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:42 smithi012 ceph-mon[38187]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:42.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:42 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2406599162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:15:42.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:42 smithi189 ceph-mon[30490]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:42.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:42 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2406599162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:15:44.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:43 smithi012 ceph-mon[25528]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:44.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:43 smithi012 ceph-mon[38187]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:44.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:43 smithi189 ceph-mon[30490]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:45.846 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:45 smithi012 ceph-mon[38187]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 59 KiB/s wr, 5 op/s 2024-09-16T14:15:45.846 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:45 smithi012 ceph-mon[25528]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 59 KiB/s wr, 5 op/s 2024-09-16T14:15:46.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:45 smithi189 ceph-mon[30490]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 59 KiB/s wr, 5 op/s 2024-09-16T14:15:48.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:47 smithi012 ceph-mon[25528]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 40 KiB/s wr, 3 op/s 2024-09-16T14:15:48.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:47 smithi012 ceph-mon[38187]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 40 KiB/s wr, 3 op/s 2024-09-16T14:15:48.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:47 smithi189 ceph-mon[30490]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 40 KiB/s wr, 3 op/s 2024-09-16T14:15:49.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:48 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T14:15:49.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:48 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:49.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:48 smithi012 ceph-mon[38187]: Deploying daemon osd.3 on smithi012 2024-09-16T14:15:49.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T14:15:49.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:48 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:49.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:48 smithi012 ceph-mon[25528]: Deploying daemon osd.3 on smithi012 2024-09-16T14:15:49.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:48 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T14:15:49.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:48 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:49.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:48 smithi189 ceph-mon[30490]: Deploying daemon osd.3 on smithi012 2024-09-16T14:15:50.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:49 smithi012 ceph-mon[25528]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-09-16T14:15:50.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:49 smithi012 ceph-mon[38187]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-09-16T14:15:50.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:49 smithi189 ceph-mon[30490]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-09-16T14:15:51.944 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:51 smithi012 ceph-mon[38187]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 879 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-16T14:15:51.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:51 smithi012 ceph-mon[25528]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 879 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-16T14:15:52.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:51 smithi189 ceph-mon[30490]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 879 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-16T14:15:53.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:53.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:53.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:53.164 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:53.164 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:53.164 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:53.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:53.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:53.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:54.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:53 smithi189 ceph-mon[30490]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T14:15:54.154 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:53 smithi012 ceph-mon[25528]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T14:15:54.154 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:53 smithi012 ceph-mon[38187]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T14:15:56.232 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-09-16T14:15:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T14:15:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T14:15:56.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:56.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:56 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.818 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T14:15:56.820 INFO:tasks.cephadm:Deploying osd.4 on smithi189 with /dev/nvme1n1... 2024-09-16T14:15:56.820 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:56 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:15:57.024 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:15:58.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[25528]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T14:15:58.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[25528]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T14:15:58.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:58.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[25528]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:58.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[25528]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:58.115 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[38187]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T14:15:58.115 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[38187]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T14:15:58.115 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:58.115 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[38187]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:58.115 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:57 smithi012 ceph-mon[38187]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:58.145 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:57 smithi189 ceph-mon[30490]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T14:15:58.145 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:57 smithi189 ceph-mon[30490]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T14:15:58.145 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:57 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:58.145 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:57 smithi189 ceph-mon[30490]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-09-16T14:15:58.145 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:57 smithi189 ceph-mon[30490]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:15:58.527 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:15:58.335+0000 7f7c4e164640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T14:15:58.528 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:15:58.341+0000 7f7c45f85640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:15:58.780 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/nvme1n1 2024-09-16T14:15:58.781 INFO:teuthology.orchestra.run.smithi189.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:15:58.785 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-16T14:15:58.800 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-16T14:15:58.800 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-16T14:15:58.800 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130842 s, 801 MB/s 2024-09-16T14:15:58.800 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-16T14:15:59.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:58 smithi189 ceph-mon[30490]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:59.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:58 smithi189 ceph-mon[30490]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T14:15:59.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:59.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:59.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:58 smithi189 ceph-mon[30490]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[25528]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[25528]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[25528]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[38187]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-16T14:15:59.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[38187]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T14:15:59.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:59.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:15:59.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:58 smithi012 ceph-mon[38187]: from='osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127]' entity='osd.3' 2024-09-16T14:15:59.375 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:/dev/nvme1n1 2024-09-16T14:15:59.612 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127] boot 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:16:00.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[25528]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127] boot 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:16:00.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:15:59 smithi012 ceph-mon[38187]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: osd.3 [v2:172.21.15.12:6826/3724297127,v1:172.21.15.12:6827/3724297127] boot 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:16:00.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:15:59 smithi189 ceph-mon[30490]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi012 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi012 to 4017M 2024-09-16T14:16:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.339 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:01 smithi189 ceph-mon[30490]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T14:16:01.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi012 2024-09-16T14:16:01.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi012 to 4017M 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[25528]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi012 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi012 to 4017M 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:01.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:01 smithi012 ceph-mon[38187]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T14:16:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:02 smithi189 ceph-mon[30490]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:02 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:02 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:02 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:02.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:02 smithi189 ceph-mon[30490]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:02.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[25528]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:02.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:02.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:02.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:02.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[25528]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:02.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[38187]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:02.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:02.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:02.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:02.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:02 smithi012 ceph-mon[38187]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:04.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/3576087150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]: dispatch 2024-09-16T14:16:04.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]: dispatch 2024-09-16T14:16:04.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[38187]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]': finished 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[38187]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/3576087150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]: dispatch 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]: dispatch 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[25528]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]': finished 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[25528]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T14:16:04.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:03 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:04.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:03 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/3576087150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]: dispatch 2024-09-16T14:16:04.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:03 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]: dispatch 2024-09-16T14:16:04.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:03 smithi189 ceph-mon[30490]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:04.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:03 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53a64bdc-6216-46ad-be7b-9e0b30935829"}]': finished 2024-09-16T14:16:04.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:03 smithi189 ceph-mon[30490]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T14:16:04.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:03 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:05.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:04 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/2824226879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:05.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:04 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/2824226879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:05.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:04 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/2824226879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:06.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:05 smithi012 ceph-mon[38187]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:06.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:05 smithi012 ceph-mon[25528]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:06.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:05 smithi189 ceph-mon[30490]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:08.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:07 smithi012 ceph-mon[38187]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:08.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:07 smithi012 ceph-mon[25528]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:08.071 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:07 smithi189 ceph-mon[30490]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:10.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:09 smithi012 ceph-mon[38187]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:10.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:09 smithi012 ceph-mon[25528]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:10.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:09 smithi189 ceph-mon[30490]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:10.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T14:16:10.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:10.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:10 smithi189 ceph-mon[30490]: Deploying daemon osd.4 on smithi189 2024-09-16T14:16:11.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T14:16:11.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:11.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:10 smithi012 ceph-mon[38187]: Deploying daemon osd.4 on smithi189 2024-09-16T14:16:11.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T14:16:11.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:11.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:10 smithi012 ceph-mon[25528]: Deploying daemon osd.4 on smithi189 2024-09-16T14:16:12.016 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:11 smithi189 ceph-mon[30490]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:12.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:11 smithi012 ceph-mon[38187]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:12.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:11 smithi012 ceph-mon[25528]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:14.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:13 smithi012 ceph-mon[38187]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:14.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:13 smithi012 ceph-mon[25528]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:14.069 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:13 smithi189 ceph-mon[30490]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:15.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:15 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:15.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:15 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:15.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:15 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:15.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:15 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:15.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:15 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:15.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:15 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:15.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:15 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:15.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:15 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:15.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:15 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:16.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:16 smithi189 ceph-mon[30490]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:16.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:16 smithi012 ceph-mon[38187]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:16.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:16 smithi012 ceph-mon[25528]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:17.963 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:17 smithi189 ceph-mon[30490]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:17 smithi012 ceph-mon[38187]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:18.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:17 smithi012 ceph-mon[25528]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:18.263 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 4 on host 'smithi189' 2024-09-16T14:16:18.889 DEBUG:teuthology.orchestra.run.smithi189:osd.4> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T14:16:18.893 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/nvme2n1... 2024-09-16T14:16:18.893 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-16T14:16:19.129 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:16:19.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[38187]: from='osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T14:16:19.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[38187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T14:16:19.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[25528]: from='osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T14:16:19.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[25528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T14:16:19.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:19 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.570 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:19 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.570 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:19 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.571 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:19 smithi189 ceph-mon[30490]: from='osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T14:16:19.571 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:19 smithi189 ceph-mon[30490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T14:16:19.571 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:19 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:19.571 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:19 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:20.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[38187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T14:16:20.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[38187]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T14:16:20.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:20.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[38187]: from='osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:20.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[38187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:20.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[38187]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:20.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[25528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T14:16:20.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[25528]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T14:16:20.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:20.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[25528]: from='osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:20.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[25528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:20.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:20 smithi012 ceph-mon[25528]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:20.573 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-mon[30490]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T14:16:20.573 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-mon[30490]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T14:16:20.573 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:20.573 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-mon[30490]: from='osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:20.573 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-mon[30490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:20.573 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-mon[30490]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:20.838 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:16:20.743+0000 7f8dbf309640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T14:16:20.838 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:16:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:16:20.747+0000 7f8db712a640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:16:21.400 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:21 smithi189 ceph-mon[30490]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:16:21.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:21 smithi189 ceph-mon[30490]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T14:16:21.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:21.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:21.401 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:21 smithi189 ceph-mon[30490]: from='osd.4 ' entity='osd.4' 2024-09-16T14:16:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[38187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:16:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[38187]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T14:16:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:21.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[38187]: from='osd.4 ' entity='osd.4' 2024-09-16T14:16:21.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[25528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:16:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[25528]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T14:16:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:21.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:21 smithi012 ceph-mon[25528]: from='osd.4 ' entity='osd.4' 2024-09-16T14:16:22.425 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/nvme2n1 2024-09-16T14:16:22.425 INFO:teuthology.orchestra.run.smithi189.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:16:22.430 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-16T14:16:22.446 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-16T14:16:22.446 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-16T14:16:22.446 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133764 s, 784 MB/s 2024-09-16T14:16:22.446 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-16T14:16:22.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:22 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:16:22.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:22 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:16:22.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:22 smithi189 ceph-mon[30490]: osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529] boot 2024-09-16T14:16:22.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:22 smithi189 ceph-mon[30490]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T14:16:22.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:22.550 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:22 smithi189 ceph-mon[30490]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:16:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:16:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[25528]: osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529] boot 2024-09-16T14:16:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[25528]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[25528]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[38187]: osd.4 [v2:172.21.15.189:6800/2190471529,v1:172.21.15.189:6801/2190471529] boot 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[38187]: osdmap e30: 5 total, 5 up, 5 in 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:16:22.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:22 smithi012 ceph-mon[38187]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T14:16:23.509 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:/dev/nvme2n1 2024-09-16T14:16:23.532 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T14:16:23.532 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.532 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.532 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:23.533 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.699 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:23.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:23.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:23.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:23.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:23.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:23.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:24.461 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:24 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi189 2024-09-16T14:16:24.461 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:24 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-16T14:16:24.461 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:24 smithi189 ceph-mon[30490]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T14:16:24.461 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:24 smithi189 ceph-mon[30490]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:24.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi189 2024-09-16T14:16:24.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-16T14:16:24.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[25528]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T14:16:24.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[25528]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:24.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi189 2024-09-16T14:16:24.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-16T14:16:24.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[38187]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T14:16:24.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:24 smithi012 ceph-mon[38187]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:25.587 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:25 smithi189 ceph-mon[30490]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T14:16:25.678 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:25 smithi012 ceph-mon[38187]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T14:16:25.678 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:25 smithi012 ceph-mon[25528]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T14:16:26.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:26 smithi189 ceph-mon[30490]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T14:16:26.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:26 smithi189 ceph-mon[30490]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:26.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:26.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:26.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:26 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:26.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:26 smithi189 ceph-mon[30490]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T14:16:26.698 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[25528]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T14:16:26.698 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[25528]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:26.698 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:26.698 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:26.698 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:26.698 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[25528]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T14:16:26.699 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[38187]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T14:16:26.699 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[38187]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:26.699 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:26.699 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:26.699 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:26.699 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:26 smithi012 ceph-mon[38187]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T14:16:27.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/3795457737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]: dispatch 2024-09-16T14:16:27.940 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]: dispatch 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]': finished 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[38187]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/3795457737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]: dispatch 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]: dispatch 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]': finished 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[25528]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T14:16:27.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:27 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:28.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:27 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/3795457737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]: dispatch 2024-09-16T14:16:28.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:27 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]: dispatch 2024-09-16T14:16:28.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:27 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4304af0-9c00-4c22-b320-977e69fe0bb4"}]': finished 2024-09-16T14:16:28.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:27 smithi189 ceph-mon[30490]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T14:16:28.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:27 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:28.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:28 smithi189 ceph-mon[30490]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 144 MiB used, 447 GiB / 447 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-16T14:16:29.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:28 smithi012 ceph-mon[25528]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 144 MiB used, 447 GiB / 447 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-16T14:16:29.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:28 smithi012 ceph-mon[38187]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 144 MiB used, 447 GiB / 447 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-16T14:16:30.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:29 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/2518604040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:30.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:29 smithi012 ceph-mon[25528]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:16:30.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:29 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/2518604040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:30.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:29 smithi012 ceph-mon[38187]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:16:30.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:29 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/2518604040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:30.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:29 smithi189 ceph-mon[30490]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:16:32.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:31 smithi012 ceph-mon[25528]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T14:16:32.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:31 smithi012 ceph-mon[38187]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T14:16:32.051 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:31 smithi189 ceph-mon[30490]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-16T14:16:33.980 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:33 smithi189 ceph-mon[30490]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:34.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:33 smithi012 ceph-mon[25528]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:34.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:33 smithi012 ceph-mon[38187]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:35.008 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T14:16:35.009 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:35.009 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:34 smithi189 ceph-mon[30490]: Deploying daemon osd.5 on smithi189 2024-09-16T14:16:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T14:16:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:34 smithi012 ceph-mon[25528]: Deploying daemon osd.5 on smithi189 2024-09-16T14:16:35.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T14:16:35.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:35.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:34 smithi012 ceph-mon[38187]: Deploying daemon osd.5 on smithi189 2024-09-16T14:16:35.957 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:35 smithi189 ceph-mon[30490]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:36.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:35 smithi012 ceph-mon[25528]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:36.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:35 smithi012 ceph-mon[38187]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:37.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:37 smithi189 ceph-mon[30490]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:38.006 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:37 smithi012 ceph-mon[38187]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:38.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:37 smithi012 ceph-mon[25528]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:39 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:39 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:39 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:40.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:39 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:40.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:39 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:40.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:39 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:40.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:39 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:40.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:39 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:40.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:39 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:40.761 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:40 smithi189 ceph-mon[30490]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:41.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:40 smithi012 ceph-mon[25528]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:41.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:40 smithi012 ceph-mon[38187]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:41.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:41 smithi189 ceph-mon[30490]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:42.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:41 smithi012 ceph-mon[25528]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:42.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:41 smithi012 ceph-mon[38187]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:42.194 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-09-16T14:16:43.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:43 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:43.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:43 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:43.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:43 smithi189 ceph-mon[30490]: from='osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T14:16:43.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:43 smithi189 ceph-mon[30490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T14:16:43.221 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T14:16:43.263 INFO:tasks.cephadm:Deploying osd.6 on smithi189 with /dev/nvme3n1... 2024-09-16T14:16:43.264 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-16T14:16:43.504 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[25528]: from='osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[25528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[38187]: from='osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T14:16:43.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:43 smithi012 ceph-mon[38187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T14:16:44.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T14:16:44.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T14:16:44.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:44.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: from='osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[25528]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: from='osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:44.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:44.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:44.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:44 smithi012 ceph-mon[38187]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:44.530 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T14:16:44.530 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T14:16:44.530 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:44.530 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: from='osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:44.530 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:16:44.531 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:44.531 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:44.531 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-mon[30490]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T14:16:44.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:16:44.658+0000 7f323ded6640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T14:16:44.838 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:16:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:16:44.663+0000 7f3235cf7640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: from='osd.5 ' entity='osd.5' 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370] boot 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T14:16:45.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: from='osd.5 ' entity='osd.5' 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370] boot 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T14:16:45.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:45 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: from='osd.5 ' entity='osd.5' 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: osd.5 [v2:172.21.15.189:6808/536159370,v1:172.21.15.189:6809/536159370] boot 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T14:16:45.566 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:45 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:16:45.894 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/nvme3n1 2024-09-16T14:16:45.894 INFO:teuthology.orchestra.run.smithi189.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:16:45.898 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-16T14:16:45.914 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-16T14:16:45.914 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-16T14:16:45.914 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140533 s, 746 MB/s 2024-09-16T14:16:45.914 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-16T14:16:46.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:16:46.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:16:46.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[25528]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:46.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[25528]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T14:16:46.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:16:46.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:16:46.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[38187]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:46.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:46 smithi012 ceph-mon[38187]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T14:16:46.581 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:46 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:16:46.581 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:46 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:16:46.581 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:46 smithi189 ceph-mon[30490]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:46.581 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:46 smithi189 ceph-mon[30490]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T14:16:46.886 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:/dev/nvme3n1 2024-09-16T14:16:47.399 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:16:48.468 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:48 smithi189 ceph-mon[30490]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T14:16:48.468 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:48 smithi189 ceph-mon[30490]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:48 smithi012 ceph-mon[25528]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T14:16:48.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:48 smithi012 ceph-mon[25528]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:48.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:48 smithi012 ceph-mon[38187]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T14:16:48.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:48 smithi012 ceph-mon[38187]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:49.448 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:49 smithi189 ceph-mon[30490]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T14:16:49.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:49 smithi012 ceph-mon[25528]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T14:16:49.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:49 smithi012 ceph-mon[38187]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T14:16:50.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi189 2024-09-16T14:16:50.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T14:16:50.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:50.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:50.504 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:50 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi189 2024-09-16T14:16:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T14:16:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:50.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi189 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:50.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:16:50.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:16:50.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:16:50.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:16:50.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:16:50.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:50 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:51.458 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:51 smithi189 ceph-mon[30490]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:51 smithi012 ceph-mon[38187]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:51.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:51 smithi012 ceph-mon[25528]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:16:52.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[25528]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-16T14:16:52.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/3772494649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]: dispatch 2024-09-16T14:16:52.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]: dispatch 2024-09-16T14:16:52.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]': finished 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[25528]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[38187]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/3772494649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]: dispatch 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]: dispatch 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]': finished 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[38187]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T14:16:52.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:16:52.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:52 smithi189 ceph-mon[30490]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-16T14:16:52.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:52 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/3772494649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]: dispatch 2024-09-16T14:16:52.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:52 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]: dispatch 2024-09-16T14:16:52.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:52 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b04497a6-0ce7-4980-98db-c573fe8c3469"}]': finished 2024-09-16T14:16:52.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:52 smithi189 ceph-mon[30490]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T14:16:52.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:16:53.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:53 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/3771456093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:53.781 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:53 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/3771456093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:53.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:53 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/3771456093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:16:54.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:54 smithi012 ceph-mon[25528]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:16:54.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:54 smithi012 ceph-mon[38187]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:16:54.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:54 smithi189 ceph-mon[30490]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:16:55.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:55 smithi189 ceph-mon[30490]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:16:56.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:55 smithi012 ceph-mon[25528]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:16:56.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:55 smithi012 ceph-mon[38187]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:16:57.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:57 smithi189 ceph-mon[30490]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:16:58.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:57 smithi012 ceph-mon[38187]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:16:58.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:57 smithi012 ceph-mon[25528]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:16:59.268 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T14:16:59.269 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:59.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T14:16:59.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:16:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T14:16:59.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:00.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:59 smithi012 ceph-mon[38187]: Deploying daemon osd.6 on smithi189 2024-09-16T14:17:00.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:16:59 smithi012 ceph-mon[38187]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:00.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:59 smithi012 ceph-mon[25528]: Deploying daemon osd.6 on smithi189 2024-09-16T14:17:00.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:16:59 smithi012 ceph-mon[25528]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:00.279 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:59 smithi189 ceph-mon[30490]: Deploying daemon osd.6 on smithi189 2024-09-16T14:17:00.279 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:16:59 smithi189 ceph-mon[30490]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:02.017 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:01 smithi189 ceph-mon[30490]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:01 smithi012 ceph-mon[25528]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:02.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:01 smithi012 ceph-mon[38187]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:04.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:03 smithi012 ceph-mon[25528]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:04.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:03 smithi012 ceph-mon[38187]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:04.033 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:03 smithi189 ceph-mon[30490]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:05.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:05.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:05.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:05.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:05.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:05.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:05.030 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:05.030 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:05.030 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:06.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:05 smithi012 ceph-mon[25528]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:06.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:05 smithi012 ceph-mon[38187]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:06.029 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:05 smithi189 ceph-mon[30490]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:07.902 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 6 on host 'smithi189' 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T14:17:08.231 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:08.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:08.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:08 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:08.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:08.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:08.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T14:17:08.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T14:17:08.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:08.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:08.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:08 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:08.561 DEBUG:teuthology.orchestra.run.smithi189:osd.6> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T14:17:08.563 INFO:tasks.cephadm:Deploying osd.7 on smithi189 with /dev/nvme4n1... 2024-09-16T14:17:08.563 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-16T14:17:09.275 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:09.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-mon[30490]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T14:17:09.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-mon[30490]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T14:17:09.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:09.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-mon[30490]: from='osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:09.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-mon[30490]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:09.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[25528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T14:17:09.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[25528]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T14:17:09.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:09.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[25528]: from='osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:09.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[25528]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[38187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T14:17:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[38187]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T14:17:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[38187]: from='osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:09.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:09 smithi012 ceph-mon[38187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:09.714 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:17:09.613+0000 7f607dc1f640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T14:17:09.714 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:17:09 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:17:09.621+0000 7f6075a40640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:17:10.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:10 smithi189 ceph-mon[30490]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:17:10.300 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:10 smithi189 ceph-mon[30490]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T14:17:10.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:10.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:10 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:10.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:10 smithi189 ceph-mon[30490]: from='osd.6 ' entity='osd.6' 2024-09-16T14:17:10.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:10 smithi189 ceph-mon[30490]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[25528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:17:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[25528]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T14:17:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:10.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[25528]: from='osd.6 ' entity='osd.6' 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[25528]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[38187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[38187]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[38187]: from='osd.6 ' entity='osd.6' 2024-09-16T14:17:10.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:10 smithi012 ceph-mon[38187]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T14:17:11.291 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:11 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:17:11.291 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:11 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:17:11.291 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:11 smithi189 ceph-mon[30490]: osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990] boot 2024-09-16T14:17:11.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:11 smithi189 ceph-mon[30490]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T14:17:11.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:11 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:11.292 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:11 smithi189 ceph-mon[30490]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T14:17:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:17:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:17:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[38187]: osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990] boot 2024-09-16T14:17:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[38187]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T14:17:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:11.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[38187]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T14:17:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:17:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:17:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[25528]: osd.6 [v2:172.21.15.189:6816/508101990,v1:172.21.15.189:6817/508101990] boot 2024-09-16T14:17:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[25528]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T14:17:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:17:11.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:11 smithi012 ceph-mon[25528]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T14:17:11.962 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/nvme4n1 2024-09-16T14:17:11.962 INFO:teuthology.orchestra.run.smithi189.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T14:17:11.967 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-16T14:17:11.984 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-16T14:17:11.984 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-16T14:17:11.984 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151727 s, 691 MB/s 2024-09-16T14:17:11.984 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-16T14:17:12.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:12 smithi189 ceph-mon[30490]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T14:17:12.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:12 smithi189 ceph-mon[30490]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 597 MiB used, 625 GiB / 626 GiB avail 2024-09-16T14:17:12.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:12 smithi012 ceph-mon[38187]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T14:17:12.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:12 smithi012 ceph-mon[38187]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 597 MiB used, 625 GiB / 626 GiB avail 2024-09-16T14:17:12.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:12 smithi012 ceph-mon[25528]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T14:17:12.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:12 smithi012 ceph-mon[25528]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 597 MiB used, 625 GiB / 626 GiB avail 2024-09-16T14:17:12.643 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:/dev/nvme4n1 2024-09-16T14:17:13.236 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:13 smithi189 ceph-mon[30490]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T14:17:13.349 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:13.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:13 smithi012 ceph-mon[25528]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T14:17:13.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:13 smithi012 ceph-mon[38187]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi189 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.518 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-16T14:17:14.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:14.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:14.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.519 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:14 smithi189 ceph-mon[30490]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 597 MiB used, 625 GiB / 626 GiB avail 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi189 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-16T14:17:14.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[38187]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 597 MiB used, 625 GiB / 626 GiB avail 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi189 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-16T14:17:14.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:14.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:14.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:14.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:14 smithi012 ceph-mon[25528]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 597 MiB used, 625 GiB / 626 GiB avail 2024-09-16T14:17:15.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:15 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:17:15.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:15 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:17:15.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:15 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:15.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:15 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:17:15.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:15 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:17:15.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:15 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:15.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:15 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T14:17:15.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:15 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T14:17:15.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:15 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:16.578 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:16 smithi189 ceph-mon[30490]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:16.578 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:16 smithi189 ceph-mon[30490]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-16T14:17:16.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:16 smithi012 ceph-mon[38187]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:16.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:16 smithi012 ceph-mon[38187]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-16T14:17:16.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:16 smithi012 ceph-mon[25528]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:16.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:16 smithi012 ceph-mon[25528]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/588525287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]: dispatch 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]: dispatch 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[38187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]': finished 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[38187]: osdmap e51: 8 total, 7 up, 8 in 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/588525287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]: dispatch 2024-09-16T14:17:18.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]: dispatch 2024-09-16T14:17:18.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[25528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]': finished 2024-09-16T14:17:18.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[25528]: osdmap e51: 8 total, 7 up, 8 in 2024-09-16T14:17:18.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:17 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:17 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/588525287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]: dispatch 2024-09-16T14:17:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:17 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]: dispatch 2024-09-16T14:17:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:17 smithi189 ceph-mon[30490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f0bdd74-b9d6-415f-87b4-e236dcca7113"}]': finished 2024-09-16T14:17:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:17 smithi189 ceph-mon[30490]: osdmap e51: 8 total, 7 up, 8 in 2024-09-16T14:17:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:17 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:18.827 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:18 smithi189 ceph-mon[30490]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:17:19.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:18 smithi012 ceph-mon[25528]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:17:19.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:18 smithi012 ceph-mon[38187]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:17:20.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:19 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/3403844388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:17:20.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:19 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/3403844388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:17:20.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:19 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/3403844388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T14:17:20.966 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:20 smithi189 ceph-mon[30490]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:17:21.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:20 smithi012 ceph-mon[25528]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:17:21.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:20 smithi012 ceph-mon[38187]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:17:22.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:21 smithi012 ceph-mon[38187]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:17:22.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:21 smithi012 ceph-mon[25528]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:17:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:21 smithi189 ceph-mon[30490]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:17:24.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:23 smithi012 ceph-mon[25528]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:23 smithi012 ceph-mon[38187]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:23 smithi189 ceph-mon[30490]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:24.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:24 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T14:17:24.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:24 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:24.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:24 smithi189 ceph-mon[30490]: Deploying daemon osd.7 on smithi189 2024-09-16T14:17:25.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:24 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T14:17:25.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:24 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:25.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:24 smithi012 ceph-mon[38187]: Deploying daemon osd.7 on smithi189 2024-09-16T14:17:25.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:24 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T14:17:25.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:24 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:25.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:24 smithi012 ceph-mon[25528]: Deploying daemon osd.7 on smithi189 2024-09-16T14:17:26.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:25 smithi012 ceph-mon[25528]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:26.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:25 smithi012 ceph-mon[38187]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:26.071 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:25 smithi189 ceph-mon[30490]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:27.997 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:27 smithi189 ceph-mon[30490]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:28.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:27 smithi012 ceph-mon[25528]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:28.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:27 smithi012 ceph-mon[38187]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:30.021 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:29 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:30.021 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:29 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:30.021 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:29 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:30.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:29 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:30.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:29 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:30.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:29 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:30.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:29 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:30.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:29 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:30.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:29 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:30.960 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:30 smithi189 ceph-mon[30490]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:31.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:30 smithi012 ceph-mon[25528]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:31.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:30 smithi012 ceph-mon[38187]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:32.751 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:32 smithi189 ceph-mon[30490]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:32.798 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 7 on host 'smithi189' 2024-09-16T14:17:33.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:32 smithi012 ceph-mon[25528]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:33.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:32 smithi012 ceph-mon[38187]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:33.456 DEBUG:teuthology.orchestra.run.smithi189:osd.7> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T14:17:33.462 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-16T14:17:33.462 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T14:17:33.680 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[25528]: from='osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[25528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[38187]: from='osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[38187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:33.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:33 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:34.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:33 smithi189 ceph-mon[30490]: from='osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T14:17:34.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:33 smithi189 ceph-mon[30490]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T14:17:34.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:33 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:34.028 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:33 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T14:17:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T14:17:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: from='osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:35.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: from='osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:34 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: from='osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-16T14:17:35.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:34 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:35.382 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:17:35.184+0000 7f6e3c1b7640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T14:17:35.382 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:17:35.191+0000 7f6e33fd8640 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:17:35.681 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.958 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: from='osd.7 ' entity='osd.7' 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1862576154' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: from='osd.7 ' entity='osd.7' 2024-09-16T14:17:35.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:35 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1862576154' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T14:17:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-16T14:17:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T14:17:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T14:17:36.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: from='osd.7 ' entity='osd.7' 2024-09-16T14:17:36.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:35 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1862576154' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T14:17:36.249 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":54,"num_osds":8,"num_up_osds":7,"osd_up_since":1726496230,"num_in_osds":8,"osd_in_since":1726496237,"num_remapped_pgs":1} 2024-09-16T14:17:36.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: purged_snaps scrub starts 2024-09-16T14:17:36.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: purged_snaps scrub ok 2024-09-16T14:17:36.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:36.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196] boot 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: purged_snaps scrub starts 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: purged_snaps scrub ok 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196] boot 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T14:17:36.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:36 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: purged_snaps scrub starts 2024-09-16T14:17:36.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: purged_snaps scrub ok 2024-09-16T14:17:36.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-16T14:17:36.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:36.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: osd.7 [v2:172.21.15.189:6824/2403843196,v1:172.21.15.189:6825/2403843196] boot 2024-09-16T14:17:36.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T14:17:36.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:36 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:17:37.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T14:17:37.457 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: Detected new or changed devices on smithi189 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-16T14:17:38.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:38.529 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: Detected new or changed devices on smithi189 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:38.530 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:38.531 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:38 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T14:17:38.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:38.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: Detected new or changed devices on smithi189 2024-09-16T14:17:38.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:38.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:38 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:39.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:17:39.455 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:39 smithi012 ceph-mon[38187]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T14:17:39.455 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:39 smithi012 ceph-mon[25528]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T14:17:39.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:39 smithi189 ceph-mon[30490]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T14:17:40.056 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1726496256,"num_in_osds":8,"osd_in_since":1726496237,"num_remapped_pgs":0} 2024-09-16T14:17:40.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T14:17:40.266 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:17:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:40 smithi012 ceph-mon[38187]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T14:17:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:40 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/4166811457' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T14:17:40.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:40 smithi012 ceph-mon[38187]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:40.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:40 smithi012 ceph-mon[25528]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T14:17:40.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:40 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/4166811457' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T14:17:40.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:40 smithi012 ceph-mon[25528]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:40.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:40 smithi189 ceph-mon[30490]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T14:17:40.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:40 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/4166811457' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T14:17:40.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:40 smithi189 ceph-mon[30490]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:41.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:41 smithi012 ceph-mon[25528]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T14:17:41.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:41 smithi012 ceph-mon[38187]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T14:17:41.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:41 smithi189 ceph-mon[30490]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T14:17:41.830 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:17:41.830 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"fsid":"9a9578d0-7435-11ef-bceb-c7b262605968","created":"2024-09-16T14:12:24.489893+0000","modified":"2024-09-16T14:17:40.166420+0000","last_up_change":"2024-09-16T14:17:36.153394+0000","last_in_change":"2024-09-16T14:17:17.589097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:15:34.108043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc317e5-4d35-45ec-9771-6cbee7431be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6803","nonce":1318340591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6805","nonce":1318340591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6809","nonce":1318340591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6807","nonce":1318340591}]},"public_addr":"172.21.15.12:6803/1318340591","cluster_addr":"172.21.15.12:6805/1318340591","heartbeat_back_addr":"172.21.15.12:6809/1318340591","heartbeat_front_addr":"172.21.15.12:6807/1318340591","state":["exists","up"]},{"osd":1,"uuid":"c5dfe945-9adc-4b48-aad6-adb6427ebe6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6811","nonce":4046669740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6813","nonce":4046669740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6817","nonce":4046669740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6815","nonce":4046669740}]},"public_addr":"172.21.15.12:6811/4046669740","cluster_addr":"172.21.15.12:6813/4046669740","heartbeat_back_addr":"172.21.15.12:6817/4046669740","heartbeat_front_addr":"172.21.15.12:6815/4046669740","state":["exists","up"]},{"osd":2,"uuid":"37dca01c-8cbf-44df-ac06-f83d9ab1143f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6819","nonce":759885517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6821","nonce":759885517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6825","nonce":759885517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6823","nonce":759885517}]},"public_addr":"172.21.15.12:6819/759885517","cluster_addr":"172.21.15.12:6821/759885517","heartbeat_back_addr":"172.21.15.12:6825/759885517","heartbeat_front_addr":"172.21.15.12:6823/759885517","state":["exists","up"]},{"osd":3,"uuid":"98f70795-ebfd-4724-b972-843fdbfe69bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6827","nonce":3724297127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6829","nonce":3724297127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6833","nonce":3724297127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6831","nonce":3724297127}]},"public_addr":"172.21.15.12:6827/3724297127","cluster_addr":"172.21.15.12:6829/3724297127","heartbeat_back_addr":"172.21.15.12:6833/3724297127","heartbeat_front_addr":"172.21.15.12:6831/3724297127","state":["exists","up"]},{"osd":4,"uuid":"53a64bdc-6216-46ad-be7b-9e0b30935829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6801","nonce":2190471529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6803","nonce":2190471529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6807","nonce":2190471529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6805","nonce":2190471529}]},"public_addr":"172.21.15.189:6801/2190471529","cluster_addr":"172.21.15.189:6803/2190471529","heartbeat_back_addr":"172.21.15.189:6807/2190471529","heartbeat_front_addr":"172.21.15.189:6805/2190471529","state":["exists","up"]},{"osd":5,"uuid":"c4304af0-9c00-4c22-b320-977e69fe0bb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6809","nonce":536159370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6811","nonce":536159370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6815","nonce":536159370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6813","nonce":536159370}]},"public_addr":"172.21.15.189:6809/536159370","cluster_addr":"172.21.15.189:6811/536159370","heartbeat_back_addr":"172.21.15.189:6815/536159370","heartbeat_front_addr":"172.21.15.189:6813/536159370","state":["exists","up"]},{"osd":6,"uuid":"b04497a6-0ce7-4980-98db-c573fe8c3469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6817","nonce":508101990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6819","nonce":508101990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6823","nonce":508101990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6821","nonce":508101990}]},"public_addr":"172.21.15.189:6817/508101990","cluster_addr":"172.21.15.189:6819/508101990","heartbeat_back_addr":"172.21.15.189:6823/508101990","heartbeat_front_addr":"172.21.15.189:6821/508101990","state":["exists","up"]},{"osd":7,"uuid":"0f0bdd74-b9d6-415f-87b4-e236dcca7113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6825","nonce":2403843196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6827","nonce":2403843196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6831","nonce":2403843196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6829","nonce":2403843196}]},"public_addr":"172.21.15.189:6825/2403843196","cluster_addr":"172.21.15.189:6827/2403843196","heartbeat_back_addr":"172.21.15.189:6831/2403843196","heartbeat_front_addr":"172.21.15.189:6829/2403843196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:14:43.264845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:07.295676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:31.629465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:56.942919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:16:19.323976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:16:43.660917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:17:08.518002+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:17:33.784487+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3205454442":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/1994445155":"2024-09-17T14:13:21.613775+0000","172.21.15.12:6801/3206746657":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/1679520617":"2024-09-17T14:13:21.613775+0000","172.21.15.12:6800/3206746657":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/393331096":"2024-09-17T14:12:58.184561+0000","172.21.15.12:0/441646478":"2024-09-17T14:12:58.184561+0000","172.21.15.12:6801/3186802043":"2024-09-17T14:12:58.184561+0000","172.21.15.12:6800/3186802043":"2024-09-17T14:12:58.184561+0000","172.21.15.12:0/714893902":"2024-09-17T14:12:58.184561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:17:42.377 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:42 smithi012 ceph-mon[38187]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:42.377 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:42 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3835859535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:17:42.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:42 smithi012 ceph-mon[25528]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:42.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:42 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3835859535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:17:42.400 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:15:34.108043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T14:17:42.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T14:17:42.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:42 smithi189 ceph-mon[30490]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:42.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:42 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3835859535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:17:42.604 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:17:43.961 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:43 smithi012 ceph-mon[38187]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:17:43.961 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:43 smithi012 ceph-mon[25528]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:17:44.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:43 smithi189 ceph-mon[30490]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T14:17:44.342 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-09-16T14:17:44.899 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:44 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/246003630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T14:17:44.899 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:44 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/246003630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T14:17:44.922 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi012 2024-09-16T14:17:44.922 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi012=foo.a' 2024-09-16T14:17:45.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:44 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/246003630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T14:17:45.132 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:46.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:45 smithi012 ceph-mon[25528]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:17:46.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:45 smithi012 ceph-mon[38187]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:17:46.087 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:45 smithi189 ceph-mon[30490]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-16T14:17:46.902 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled rgw.foo.a update... 2024-09-16T14:17:47.623 DEBUG:teuthology.orchestra.run.smithi012:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-16T14:17:47.626 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi189 2024-09-16T14:17:47.626 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-16T14:17:47.819 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi012=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: Saving service rgw.foo.a spec with placement smithi012=foo.a;count:1 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:48.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: Deploying daemon rgw.foo.a on smithi012 2024-09-16T14:17:48.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:47 smithi189 ceph-mon[30490]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi012=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: Saving service rgw.foo.a spec with placement smithi012=foo.a;count:1 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:48.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: Deploying daemon rgw.foo.a on smithi012 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[25528]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi012=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: Saving service rgw.foo.a spec with placement smithi012=foo.a;count:1 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:48.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: Deploying daemon rgw.foo.a on smithi012 2024-09-16T14:17:48.279 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:47 smithi012 ceph-mon[38187]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-16T14:17:49.277 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:17:49 smithi012 podman[72088]: 2024-09-16 14:17:49.223909255 +0000 UTC m=+0.136656111 container create ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T14:17:49.777 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:17:49 smithi012 podman[72088]: 2024-09-16 14:17:49.459952623 +0000 UTC m=+0.372699479 container init ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T14:17:49.777 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:17:49 smithi012 podman[72088]: 2024-09-16 14:17:49.46336397 +0000 UTC m=+0.376110828 container start ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T14:17:49.777 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:17:49 smithi012 bash[72088]: ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 2024-09-16T14:17:49.777 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:17:49 smithi012 systemd[1]: Started Ceph rgw.foo.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:17:50.385 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'datapool' created 2024-09-16T14:17:50.704 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:50 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/2981786793' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T14:17:50.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:50 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/2981786793' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T14:17:50.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:50 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/2981786793' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T14:17:51.008 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-16T14:17:51.201 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T14:17:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T14:17:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T14:17:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: Saving service rgw.foo.a spec with placement smithi012=foo.a;count:1 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:51 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:51.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:51.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T14:17:51.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T14:17:51.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T14:17:51.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: Saving service rgw.foo.a spec with placement smithi012=foo.a;count:1 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T14:17:51.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: Saving service rgw.foo.a spec with placement smithi012=foo.a;count:1 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:51.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:51 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:17:52.676 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T14:17:52.676 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T14:17:52.676 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: pgmap v186: 36 pgs: 3 creating+peering, 32 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:52.676 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.677 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.677 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.677 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:52.677 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:52 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:52.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T14:17:52.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T14:17:52.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: pgmap v186: 36 pgs: 3 creating+peering, 32 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:52.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: pgmap v186: 36 pgs: 3 creating+peering, 32 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:52.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:52 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: Checking dashboard <-> RGW credentials 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3298076514' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/1755895811' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[25528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: Checking dashboard <-> RGW credentials 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3298076514' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/1755895811' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T14:17:53.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:53 smithi012 ceph-mon[38187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T14:17:53.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: Checking dashboard <-> RGW credentials 2024-09-16T14:17:53.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T14:17:53.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T14:17:53.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3298076514' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T14:17:53.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/1755895811' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T14:17:53.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:53 smithi189 ceph-mon[30490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T14:17:54.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T14:17:54.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[38187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T14:17:54.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[38187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T14:17:54.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[38187]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T14:17:54.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[38187]: pgmap v189: 68 pgs: 12 active+clean, 4 creating+peering, 52 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T14:17:54.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T14:17:54.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[25528]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T14:17:54.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[25528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T14:17:54.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[25528]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T14:17:54.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:54 smithi012 ceph-mon[25528]: pgmap v189: 68 pgs: 12 active+clean, 4 creating+peering, 52 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T14:17:54.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:54 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T14:17:54.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:54 smithi189 ceph-mon[30490]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T14:17:54.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:54 smithi189 ceph-mon[30490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T14:17:54.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:54 smithi189 ceph-mon[30490]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T14:17:54.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:54 smithi189 ceph-mon[30490]: pgmap v189: 68 pgs: 12 active+clean, 4 creating+peering, 52 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: Cluster is now healthy 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3298076514' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[25528]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T14:17:55.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: Cluster is now healthy 2024-09-16T14:17:55.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T14:17:55.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3298076514' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:55 smithi012 ceph-mon[38187]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: Cluster is now healthy 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3298076514' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1157807337' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:55.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:55 smithi189 ceph-mon[30490]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T14:17:56.101 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.189 --placement '1;smithi189=iscsi.a' 2024-09-16T14:17:56.294 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:56.750 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:56 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T14:17:56.750 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:56 smithi189 ceph-mon[30490]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T14:17:56.750 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:56 smithi189 ceph-mon[30490]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T14:17:56.750 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:56 smithi189 ceph-mon[30490]: pgmap v192: 100 pgs: 50 active+clean, 20 creating+peering, 30 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-16T14:17:56.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T14:17:56.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[38187]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T14:17:56.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[38187]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T14:17:56.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[38187]: pgmap v192: 100 pgs: 50 active+clean, 20 creating+peering, 30 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-16T14:17:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T14:17:56.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[25528]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T14:17:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[25528]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T14:17:56.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:56 smithi012 ceph-mon[25528]: pgmap v192: 100 pgs: 50 active+clean, 20 creating+peering, 30 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-16T14:17:57.745 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:57 smithi189 ceph-mon[30490]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T14:17:57.746 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:57 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.746 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:57 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3583799923' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.746 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:57 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[38187]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T14:17:57.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3583799923' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[25528]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T14:17:57.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3583799923' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:57.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:57 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T14:17:58.155 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled iscsi.datapool update... 2024-09-16T14:17:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T14:17:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T14:17:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T14:17:58.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3583799923' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: pgmap v195: 132 pgs: 78 active+clean, 28 creating+peering, 26 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 511 B/s wr, 0 op/s 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T14:17:58.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T14:17:58.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3583799923' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: pgmap v195: 132 pgs: 78 active+clean, 28 creating+peering, 26 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 511 B/s wr, 0 op/s 2024-09-16T14:17:58.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:58 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3583799923' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: pgmap v195: 132 pgs: 78 active+clean, 28 creating+peering, 26 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 511 B/s wr, 0 op/s 2024-09-16T14:17:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:58 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:58.897 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-16T14:17:58.897 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:17:58.897 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T14:17:58.928 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:17:58.928 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T14:17:58.957 DEBUG:teuthology.orchestra.run.smithi189:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-16T14:17:59.000 INFO:tasks.cephadm:Adding prometheus.a on smithi189 2024-09-16T14:17:59.001 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi189=a' 2024-09-16T14:17:59.743 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T14:17:59.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T14:17:59.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T14:17:59.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:17:59 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:17:59.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:17:59.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-16T14:17:59.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2609737910' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T14:17:59.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T14:17:59.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T14:17:59.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:17:59.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T14:17:59.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T14:17:59.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:17:59 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:00.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:00 smithi189 ceph-mon[30490]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-16T14:18:00.737 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:00 smithi189 ceph-mon[30490]: pgmap v197: 132 pgs: 86 active+clean, 27 creating+peering, 19 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:00.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:00 smithi012 ceph-mon[38187]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-16T14:18:00.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:00 smithi012 ceph-mon[38187]: pgmap v197: 132 pgs: 86 active+clean, 27 creating+peering, 19 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:00.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:00 smithi012 ceph-mon[25528]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-16T14:18:00.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:00 smithi012 ceph-mon[25528]: pgmap v197: 132 pgs: 86 active+clean, 27 creating+peering, 19 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:01.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:00 smithi189 systemd[1]: Starting Ceph iscsi.iscsi.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:18:01.747 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:01 smithi189 podman[66246]: 2024-09-16 14:18:01.372527113 +0000 UTC m=+0.022486540 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:18:01.747 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:01 smithi189 podman[66246]: 2024-09-16 14:18:01.746595544 +0000 UTC m=+0.396554947 container create d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T14:18:02.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:01 smithi012 ceph-mon[25528]: pgmap v198: 132 pgs: 121 active+clean, 11 creating+peering; 581 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.3 KiB/s wr, 132 op/s 2024-09-16T14:18:02.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:01 smithi012 ceph-mon[38187]: pgmap v198: 132 pgs: 121 active+clean, 11 creating+peering; 581 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.3 KiB/s wr, 132 op/s 2024-09-16T14:18:02.058 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:01 smithi189 ceph-mon[30490]: pgmap v198: 132 pgs: 121 active+clean, 11 creating+peering; 581 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.3 KiB/s wr, 132 op/s 2024-09-16T14:18:02.253 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled prometheus update... 2024-09-16T14:18:02.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:02 smithi189 podman[66246]: 2024-09-16 14:18:02.167101378 +0000 UTC m=+0.817060796 container init d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:18:02.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:02 smithi189 podman[66246]: 2024-09-16 14:18:02.170556852 +0000 UTC m=+0.820516258 container start d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T14:18:02.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:02 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu[66299]: log file path now is '/var/log/tcmu-runner.log' 2024-09-16T14:18:02.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:02 smithi189 bash[66246]: d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 2024-09-16T14:18:03.001 DEBUG:teuthology.orchestra.run.smithi189:prometheus.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-16T14:18:03.004 INFO:tasks.cephadm:Adding node-exporter.a on smithi012 2024-09-16T14:18:03.004 INFO:tasks.cephadm:Adding node-exporter.b on smithi189 2024-09-16T14:18:03.004 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi012=a;smithi189=b' 2024-09-16T14:18:03.251 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 podman[66424]: 2024-09-16 14:18:03.099448799 +0000 UTC m=+0.025357443 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:18:03.251 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 podman[66424]: 2024-09-16 14:18:03.205064884 +0000 UTC m=+0.130973508 container create 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T14:18:03.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:03 smithi189 ceph-mon[30490]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:03.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:03 smithi189 ceph-mon[30490]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-16T14:18:03.503 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:03 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:03.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 podman[66424]: 2024-09-16 14:18:03.457750717 +0000 UTC m=+0.383659342 container init 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-16T14:18:03.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 podman[66424]: 2024-09-16 14:18:03.461386026 +0000 UTC m=+0.387294648 container start 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T14:18:03.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:03 smithi012 ceph-mon[38187]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:03.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:03 smithi012 ceph-mon[38187]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-16T14:18:03.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:03 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:03.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:03 smithi012 ceph-mon[25528]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:03.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:03 smithi012 ceph-mon[25528]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-16T14:18:03.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:03 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:03.612 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:18:03.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 bash[66424]: 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 2024-09-16T14:18:03.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 systemd[1]: Started Ceph iscsi.iscsi.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:18:04.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug Started the configuration object watcher 2024-09-16T14:18:04.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug Checking for config object changes every 1s 2024-09-16T14:18:04.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug Processing osd blocklist entries for this node 2024-09-16T14:18:04.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug Reading the configuration object to update local LIO configuration 2024-09-16T14:18:04.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug Configuration does not have an entry for this host(smithi189.front.sepia.ceph.com) - nothing to define to LIO 2024-09-16T14:18:04.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-16T14:18:04.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: * Environment: production 2024-09-16T14:18:04.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T14:18:04.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: Use a production WSGI server instead. 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: * Debug mode: off 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug * Running on all addresses. 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: * Running on all addresses. 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug * Running on http://[fe80::ec4:7aff:fe88:6df1]:5000/ (Press CTRL+C to quit) 2024-09-16T14:18:04.339 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: * Running on http://[fe80::ec4:7aff:fe88:6df1]:5000/ (Press CTRL+C to quit) 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 5.2 KiB/s wr, 156 op/s 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:18:04.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:04 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/1074416367' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T14:18:05.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T14:18:05.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 5.2 KiB/s wr, 156 op/s 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/1074416367' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 5.2 KiB/s wr, 156 op/s 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:18:05.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:04 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/1074416367' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T14:18:05.572 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled node-exporter update... 2024-09-16T14:18:05.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:05 smithi189 ceph-mon[30490]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T14:18:05.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:05 smithi189 ceph-mon[30490]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T14:18:05.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:05 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:06.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:05 smithi012 ceph-mon[38187]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T14:18:06.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:05 smithi012 ceph-mon[38187]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T14:18:06.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:05 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:06.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:05 smithi012 ceph-mon[25528]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T14:18:06.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:05 smithi012 ceph-mon[25528]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T14:18:06.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:05 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:06.305 DEBUG:teuthology.orchestra.run.smithi012:node-exporter.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-16T14:18:06.308 DEBUG:teuthology.orchestra.run.smithi189:node-exporter.b> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-16T14:18:06.311 INFO:tasks.cephadm:Adding alertmanager.a on smithi012 2024-09-16T14:18:06.311 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi012=a' 2024-09-16T14:18:06.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:06 smithi189 ceph-mon[30490]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi012=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:06.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:06 smithi189 ceph-mon[30490]: Saving service node-exporter spec with placement smithi012=a;smithi189=b;count:2 2024-09-16T14:18:06.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:06 smithi189 ceph-mon[30490]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-09-16T14:18:06.881 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:18:07.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:06 smithi012 ceph-mon[38187]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi012=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:07.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:06 smithi012 ceph-mon[38187]: Saving service node-exporter spec with placement smithi012=a;smithi189=b;count:2 2024-09-16T14:18:07.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:06 smithi012 ceph-mon[38187]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-09-16T14:18:07.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:06 smithi012 ceph-mon[25528]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi012=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:07.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:06 smithi012 ceph-mon[25528]: Saving service node-exporter spec with placement smithi012=a;smithi189=b;count:2 2024-09-16T14:18:07.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:06 smithi012 ceph-mon[25528]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-09-16T14:18:07.975 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:07 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:07.975 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:07 smithi189 ceph-mon[30490]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s rd, 4.1 KiB/s wr, 123 op/s 2024-09-16T14:18:08.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:07 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:08.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:07 smithi012 ceph-mon[38187]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s rd, 4.1 KiB/s wr, 123 op/s 2024-09-16T14:18:08.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:07 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:08.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:07 smithi012 ceph-mon[25528]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s rd, 4.1 KiB/s wr, 123 op/s 2024-09-16T14:18:09.299 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled alertmanager update... 2024-09-16T14:18:09.953 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: Deploying daemon prometheus.a on smithi189 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi012=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: Saving service alertmanager spec with placement smithi012=a;count:1 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:09.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:09 smithi189 ceph-mon[30490]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.8 KiB/s wr, 113 op/s 2024-09-16T14:18:09.974 DEBUG:teuthology.orchestra.run.smithi012:alertmanager.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-16T14:18:09.977 INFO:tasks.cephadm:Adding grafana.a on smithi189 2024-09-16T14:18:09.977 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi189=a' 2024-09-16T14:18:10.197 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:18:10.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:10.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:18:10.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: Deploying daemon prometheus.a on smithi189 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi012=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: Saving service alertmanager spec with placement smithi012=a;count:1 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[38187]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.8 KiB/s wr, 113 op/s 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: Deploying daemon prometheus.a on smithi189 2024-09-16T14:18:10.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi012=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:10.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: Saving service alertmanager spec with placement smithi012=a;count:1 2024-09-16T14:18:10.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:10.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:09 smithi012 ceph-mon[25528]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.8 KiB/s wr, 113 op/s 2024-09-16T14:18:12.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:11 smithi012 ceph-mon[25528]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1.2 KiB/s wr, 34 op/s 2024-09-16T14:18:12.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:11 smithi012 ceph-mon[38187]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1.2 KiB/s wr, 34 op/s 2024-09-16T14:18:12.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:11 smithi189 ceph-mon[30490]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1.2 KiB/s wr, 34 op/s 2024-09-16T14:18:12.357 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled grafana update... 2024-09-16T14:18:13.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:13.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[38187]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:13.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[38187]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-16T14:18:13.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:13.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:13.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[25528]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:13.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[25528]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-16T14:18:13.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:12 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:13.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:12 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:13.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:12 smithi189 ceph-mon[30490]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:18:13.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:12 smithi189 ceph-mon[30490]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-16T14:18:13.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:12 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:13.549 DEBUG:teuthology.orchestra.run.smithi189:grafana.a> sudo journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@grafana.a.service 2024-09-16T14:18:13.551 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T14:18:13.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T14:18:13.768 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:13.838 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:13 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:18:14.208 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:13 smithi012 ceph-mon[38187]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-09-16T14:18:14.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:13 smithi012 ceph-mon[25528]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-09-16T14:18:14.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:13 smithi189 ceph-mon[30490]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-09-16T14:18:15.197 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:14 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:15.197 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:14 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:15.337 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:14 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:15.907 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-09-16T14:18:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAnPuhm/u+FNRAAz+toGvX0+iNl43JiWH3qXw== 2024-09-16T14:18:16.219 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:15 smithi012 ceph-mon[38187]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-16T14:18:16.220 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:15 smithi012 ceph-mon[25528]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-16T14:18:16.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:15 smithi189 ceph-mon[30490]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-16T14:18:16.555 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:18:16.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T14:18:16.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T14:18:16.598 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T14:18:17.186 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T14:18:17.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:16 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1321297373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:17.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:16 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1321297373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T14:18:17.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:16 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1321297373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:17.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:16 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1321297373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T14:18:17.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:16 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1321297373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:17.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:16 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1321297373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T14:18:18.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:17 smithi189 ceph-mon[30490]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:18.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:17 smithi012 ceph-mon[38187]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:18.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:17 smithi012 ceph-mon[25528]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:19.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:19 smithi189 ceph-mon[30490]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:20.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:19 smithi012 ceph-mon[25528]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:20.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:19 smithi012 ceph-mon[38187]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:20.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:20 smithi189 ceph-mon[30490]: from='client.? 172.21.15.189:0/2956807940' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:20.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:20 smithi189 ceph-mon[30490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:20.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:20 smithi189 ceph-mon[30490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T14:18:21.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:20 smithi012 ceph-mon[38187]: from='client.? 172.21.15.189:0/2956807940' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:21.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:20 smithi012 ceph-mon[38187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:21.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:20 smithi012 ceph-mon[38187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T14:18:21.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:20 smithi012 ceph-mon[25528]: from='client.? 172.21.15.189:0/2956807940' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:21.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:20 smithi012 ceph-mon[25528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T14:18:21.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:20 smithi012 ceph-mon[25528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T14:18:21.125 INFO:teuthology.orchestra.run.smithi189.stdout:[client.1] 2024-09-16T14:18:21.125 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAsPuhmopsgARAAVicsNWUXvbqLO1Qxp4hLwQ== 2024-09-16T14:18:21.680 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:18:21.680 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T14:18:21.681 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T14:18:21.724 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T14:18:21.724 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T14:18:21.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T14:18:21.944 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:22.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2024-09-16T14:18:22.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-16T14:18:22.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-16T14:18:22.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-16T14:18:22.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[38187]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:22.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2024-09-16T14:18:22.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-16T14:18:22.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-16T14:18:22.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-16T14:18:22.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:21 smithi012 ceph-mon[25528]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2024-09-16T14:18:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-16T14:18:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-16T14:18:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:21 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-16T14:18:22.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:21 smithi189 ceph-mon[30490]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]': finished 2024-09-16T14:18:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-16T14:18:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-16T14:18:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-16T14:18:22.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[25528]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T14:18:22.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]': finished 2024-09-16T14:18:22.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-16T14:18:22.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-16T14:18:22.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:22 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-16T14:18:22.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:22 smithi189 ceph-mon[30490]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T14:18:23.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]': finished 2024-09-16T14:18:23.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-16T14:18:23.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-16T14:18:23.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-16T14:18:23.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:22 smithi012 ceph-mon[38187]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T14:18:23.588 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:18:23 smithi189 systemd[1]: Started Ceph prometheus.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:18:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/103338443' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T14:18:24.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[38187]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/103338443' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T14:18:24.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:23 smithi012 ceph-mon[25528]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-16T14:18:24.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/103338443' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T14:18:24.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:23 smithi189 ceph-mon[30490]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-16T14:18:24.340 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":734310723},{"type":"v1","addr":"172.21.15.12:6801","nonce":734310723}]},"active_addr":"172.21.15.12:6801/734310723","active_change":"2024-09-16T14:13:21.613920+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3940402781}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3973510964}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3661537260}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3023467800}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":1343268016}]}]} 2024-09-16T14:18:24.347 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T14:18:24.347 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T14:18:24.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T14:18:24.569 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:24 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ignoring --setuser ceph since I am not root 2024-09-16T14:18:24.570 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:24 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ignoring --setgroup ceph since I am not root 2024-09-16T14:18:24.570 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:24 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:24.568+0000 7fe3b2d1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:18:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:24.758 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: ignoring --setuser ceph since I am not root 2024-09-16T14:18:24.758 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: ignoring --setgroup ceph since I am not root 2024-09-16T14:18:24.758 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-mgr[32352]: -- 172.21.15.189:0/2532396782 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5635d02991e0 con 0x5635d01ce800 2024-09-16T14:18:24.758 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-mgr[32352]: -- 172.21.15.189:0/2532396782 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5635d01fad00 con 0x5635d01ce800 2024-09-16T14:18:24.758 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:24.551+0000 7f8f79b9b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:18:24.758 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:24.608+0000 7f8f79b9b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:18:24.759 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:24.756+0000 7f8f79b9b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:18:24.861 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:24 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:24.629+0000 7fe3b2d1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:18:24.861 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:24 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:24.788+0000 7fe3b2d1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:18:24.861 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:24 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:24.859+0000 7fe3b2d1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:18:25.088 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:24.821+0000 7f8f79b9b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:18:25.088 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:24.965+0000 7f8f79b9b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:18:25.166 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:25.018+0000 7fe3b2d1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:18:25.167 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:25.165+0000 7fe3b2d1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:18:25.404 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:25.106+0000 7f8f79b9b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:18:25.495 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-mon[38187]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T14:18:25.495 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-mon[38187]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T14:18:25.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-mon[25528]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T14:18:25.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-mon[25528]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T14:18:25.661 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:25.415+0000 7f8f79b9b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:18:25.661 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:25.476+0000 7f8f79b9b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:18:25.661 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:25 smithi189 ceph-mon[30490]: from='mgr.14150 172.21.15.12:0/3111208361' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T14:18:25.661 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:25 smithi189 ceph-mon[30490]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T14:18:25.751 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:25.493+0000 7fe3b2d1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:18:25.751 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:25.557+0000 7fe3b2d1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:18:25.751 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:25 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:25.749+0000 7fe3b2d1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:18:26.088 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:25.659+0000 7f8f79b9b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:18:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:18:26.606 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"fsid":"9a9578d0-7435-11ef-bceb-c7b262605968","created":"2024-09-16T14:12:24.489893+0000","modified":"2024-09-16T14:18:22.767207+0000","last_up_change":"2024-09-16T14:17:36.153394+0000","last_in_change":"2024-09-16T14:17:17.589097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:15:34.108043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-16T14:17:49.943216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-16T14:17:50.373862+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T14:17:51.866612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T14:17:53.920826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T14:17:55.995839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc317e5-4d35-45ec-9771-6cbee7431be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6803","nonce":1318340591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6805","nonce":1318340591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6809","nonce":1318340591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6807","nonce":1318340591}]},"public_addr":"172.21.15.12:6803/1318340591","cluster_addr":"172.21.15.12:6805/1318340591","heartbeat_back_addr":"172.21.15.12:6809/1318340591","heartbeat_front_addr":"172.21.15.12:6807/1318340591","state":["exists","up"]},{"osd":1,"uuid":"c5dfe945-9adc-4b48-aad6-adb6427ebe6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6811","nonce":4046669740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6813","nonce":4046669740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6817","nonce":4046669740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6815","nonce":4046669740}]},"public_addr":"172.21.15.12:6811/4046669740","cluster_addr":"172.21.15.12:6813/4046669740","heartbeat_back_addr":"172.21.15.12:6817/4046669740","heartbeat_front_addr":"172.21.15.12:6815/4046669740","state":["exists","up"]},{"osd":2,"uuid":"37dca01c-8cbf-44df-ac06-f83d9ab1143f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6819","nonce":759885517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6821","nonce":759885517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6825","nonce":759885517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6823","nonce":759885517}]},"public_addr":"172.21.15.12:6819/759885517","cluster_addr":"172.21.15.12:6821/759885517","heartbeat_back_addr":"172.21.15.12:6825/759885517","heartbeat_front_addr":"172.21.15.12:6823/759885517","state":["exists","up"]},{"osd":3,"uuid":"98f70795-ebfd-4724-b972-843fdbfe69bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6827","nonce":3724297127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6829","nonce":3724297127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6833","nonce":3724297127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6831","nonce":3724297127}]},"public_addr":"172.21.15.12:6827/3724297127","cluster_addr":"172.21.15.12:6829/3724297127","heartbeat_back_addr":"172.21.15.12:6833/3724297127","heartbeat_front_addr":"172.21.15.12:6831/3724297127","state":["exists","up"]},{"osd":4,"uuid":"53a64bdc-6216-46ad-be7b-9e0b30935829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6801","nonce":2190471529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6803","nonce":2190471529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6807","nonce":2190471529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6805","nonce":2190471529}]},"public_addr":"172.21.15.189:6801/2190471529","cluster_addr":"172.21.15.189:6803/2190471529","heartbeat_back_addr":"172.21.15.189:6807/2190471529","heartbeat_front_addr":"172.21.15.189:6805/2190471529","state":["exists","up"]},{"osd":5,"uuid":"c4304af0-9c00-4c22-b320-977e69fe0bb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6809","nonce":536159370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6811","nonce":536159370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6815","nonce":536159370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6813","nonce":536159370}]},"public_addr":"172.21.15.189:6809/536159370","cluster_addr":"172.21.15.189:6811/536159370","heartbeat_back_addr":"172.21.15.189:6815/536159370","heartbeat_front_addr":"172.21.15.189:6813/536159370","state":["exists","up"]},{"osd":6,"uuid":"b04497a6-0ce7-4980-98db-c573fe8c3469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6817","nonce":508101990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6819","nonce":508101990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6823","nonce":508101990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6821","nonce":508101990}]},"public_addr":"172.21.15.189:6817/508101990","cluster_addr":"172.21.15.189:6819/508101990","heartbeat_back_addr":"172.21.15.189:6823/508101990","heartbeat_front_addr":"172.21.15.189:6821/508101990","state":["exists","up"]},{"osd":7,"uuid":"0f0bdd74-b9d6-415f-87b4-e236dcca7113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6825","nonce":2403843196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6827","nonce":2403843196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6831","nonce":2403843196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6829","nonce":2403843196}]},"public_addr":"172.21.15.189:6825/2403843196","cluster_addr":"172.21.15.189:6827/2403843196","heartbeat_back_addr":"172.21.15.189:6831/2403843196","heartbeat_front_addr":"172.21.15.189:6829/2403843196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:14:43.264845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:07.295676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:31.629465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:56.942919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:16:19.323976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:16:43.660917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:17:08.518002+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:17:33.784487+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":7,"to":4}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3205454442":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/1994445155":"2024-09-17T14:13:21.613775+0000","172.21.15.12:6801/3206746657":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/1679520617":"2024-09-17T14:13:21.613775+0000","172.21.15.12:6800/3206746657":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/393331096":"2024-09-17T14:12:58.184561+0000","172.21.15.12:0/441646478":"2024-09-17T14:12:58.184561+0000","172.21.15.12:6801/3186802043":"2024-09-17T14:12:58.184561+0000","172.21.15.12:6800/3186802043":"2024-09-17T14:12:58.184561+0000","172.21.15.12:0/714893902":"2024-09-17T14:12:58.184561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:18:26.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:26 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3350332462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:18:26.838 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:26.527+0000 7f8f79b9b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:18:26.838 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:26.593+0000 7f8f79b9b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:18:26.923 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:26 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3350332462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:18:26.923 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:26 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:26.694+0000 7fe3b2d1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:18:26.923 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:26 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:26.763+0000 7fe3b2d1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:18:26.923 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:26 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3350332462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:18:27.288 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T14:18:27.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T14:18:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:27.565 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.306+0000 7f8f79b9b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:18:27.565 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.421+0000 7f8f79b9b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:18:27.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:27 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:27.552+0000 7fe3b2d1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:18:27.777 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:27 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:27.673+0000 7fe3b2d1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:18:27.838 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.564+0000 7f8f79b9b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:18:27.838 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.626+0000 7f8f79b9b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:18:27.838 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.772+0000 7f8f79b9b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:18:28.049 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:27 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:27.825+0000 7fe3b2d1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:18:28.049 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:27 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:27.889+0000 7fe3b2d1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:18:28.049 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.047+0000 7fe3b2d1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:18:28.120 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.840+0000 7f8f79b9b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:18:28.120 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:27.899+0000 7f8f79b9b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:18:28.120 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:28.053+0000 7f8f79b9b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:18:28.120 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:28.118+0000 7f8f79b9b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:18:28.347 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.118+0000 7fe3b2d1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:18:28.348 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.180+0000 7fe3b2d1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:18:28.348 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.345+0000 7fe3b2d1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:18:28.384 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:28.184+0000 7f8f79b9b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:18:28.384 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:28.266+0000 7f8f79b9b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:18:28.384 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:28.324+0000 7f8f79b9b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:18:28.636 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.413+0000 7fe3b2d1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:18:28.636 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.483+0000 7fe3b2d1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:18:28.636 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.570+0000 7fe3b2d1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:18:28.636 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.632+0000 7fe3b2d1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:18:28.837 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:28.382+0000 7f8f79b9b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:18:28.911 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:28 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:28.694+0000 7fe3b2d1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[38187]: Standby manager daemon x restarted 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[38187]: Standby manager daemon x started 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[38187]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[25528]: Standby manager daemon x restarted 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[25528]: Standby manager daemon x started 2024-09-16T14:18:29.432 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T14:18:29.433 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T14:18:29.433 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T14:18:29.433 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-mon[25528]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T14:18:29.461 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:18:29.462 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"fsid":"9a9578d0-7435-11ef-bceb-c7b262605968","created":"2024-09-16T14:12:24.489893+0000","modified":"2024-09-16T14:18:22.767207+0000","last_up_change":"2024-09-16T14:17:36.153394+0000","last_in_change":"2024-09-16T14:17:17.589097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:15:34.108043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-16T14:17:49.943216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-16T14:17:50.373862+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T14:17:51.866612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T14:17:53.920826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T14:17:55.995839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc317e5-4d35-45ec-9771-6cbee7431be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6803","nonce":1318340591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6805","nonce":1318340591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6809","nonce":1318340591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1318340591},{"type":"v1","addr":"172.21.15.12:6807","nonce":1318340591}]},"public_addr":"172.21.15.12:6803/1318340591","cluster_addr":"172.21.15.12:6805/1318340591","heartbeat_back_addr":"172.21.15.12:6809/1318340591","heartbeat_front_addr":"172.21.15.12:6807/1318340591","state":["exists","up"]},{"osd":1,"uuid":"c5dfe945-9adc-4b48-aad6-adb6427ebe6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6811","nonce":4046669740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6813","nonce":4046669740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6817","nonce":4046669740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":4046669740},{"type":"v1","addr":"172.21.15.12:6815","nonce":4046669740}]},"public_addr":"172.21.15.12:6811/4046669740","cluster_addr":"172.21.15.12:6813/4046669740","heartbeat_back_addr":"172.21.15.12:6817/4046669740","heartbeat_front_addr":"172.21.15.12:6815/4046669740","state":["exists","up"]},{"osd":2,"uuid":"37dca01c-8cbf-44df-ac06-f83d9ab1143f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6819","nonce":759885517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6821","nonce":759885517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6825","nonce":759885517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":759885517},{"type":"v1","addr":"172.21.15.12:6823","nonce":759885517}]},"public_addr":"172.21.15.12:6819/759885517","cluster_addr":"172.21.15.12:6821/759885517","heartbeat_back_addr":"172.21.15.12:6825/759885517","heartbeat_front_addr":"172.21.15.12:6823/759885517","state":["exists","up"]},{"osd":3,"uuid":"98f70795-ebfd-4724-b972-843fdbfe69bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6827","nonce":3724297127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6829","nonce":3724297127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6833","nonce":3724297127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3724297127},{"type":"v1","addr":"172.21.15.12:6831","nonce":3724297127}]},"public_addr":"172.21.15.12:6827/3724297127","cluster_addr":"172.21.15.12:6829/3724297127","heartbeat_back_addr":"172.21.15.12:6833/3724297127","heartbeat_front_addr":"172.21.15.12:6831/3724297127","state":["exists","up"]},{"osd":4,"uuid":"53a64bdc-6216-46ad-be7b-9e0b30935829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6801","nonce":2190471529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6803","nonce":2190471529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6807","nonce":2190471529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2190471529},{"type":"v1","addr":"172.21.15.189:6805","nonce":2190471529}]},"public_addr":"172.21.15.189:6801/2190471529","cluster_addr":"172.21.15.189:6803/2190471529","heartbeat_back_addr":"172.21.15.189:6807/2190471529","heartbeat_front_addr":"172.21.15.189:6805/2190471529","state":["exists","up"]},{"osd":5,"uuid":"c4304af0-9c00-4c22-b320-977e69fe0bb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6809","nonce":536159370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6811","nonce":536159370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6815","nonce":536159370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":536159370},{"type":"v1","addr":"172.21.15.189:6813","nonce":536159370}]},"public_addr":"172.21.15.189:6809/536159370","cluster_addr":"172.21.15.189:6811/536159370","heartbeat_back_addr":"172.21.15.189:6815/536159370","heartbeat_front_addr":"172.21.15.189:6813/536159370","state":["exists","up"]},{"osd":6,"uuid":"b04497a6-0ce7-4980-98db-c573fe8c3469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6817","nonce":508101990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6819","nonce":508101990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6823","nonce":508101990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":508101990},{"type":"v1","addr":"172.21.15.189:6821","nonce":508101990}]},"public_addr":"172.21.15.189:6817/508101990","cluster_addr":"172.21.15.189:6819/508101990","heartbeat_back_addr":"172.21.15.189:6823/508101990","heartbeat_front_addr":"172.21.15.189:6821/508101990","state":["exists","up"]},{"osd":7,"uuid":"0f0bdd74-b9d6-415f-87b4-e236dcca7113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6825","nonce":2403843196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6827","nonce":2403843196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6831","nonce":2403843196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2403843196},{"type":"v1","addr":"172.21.15.189:6829","nonce":2403843196}]},"public_addr":"172.21.15.189:6825/2403843196","cluster_addr":"172.21.15.189:6827/2403843196","heartbeat_back_addr":"172.21.15.189:6831/2403843196","heartbeat_front_addr":"172.21.15.189:6829/2403843196","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:14:43.264845+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:07.295676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:31.629465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:15:56.942919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:16:19.323976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:16:43.660917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:17:08.518002+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:17:33.784487+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":7,"to":4}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3205454442":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/1994445155":"2024-09-17T14:13:21.613775+0000","172.21.15.12:6801/3206746657":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/1679520617":"2024-09-17T14:13:21.613775+0000","172.21.15.12:6800/3206746657":"2024-09-17T14:13:21.613775+0000","172.21.15.12:0/393331096":"2024-09-17T14:12:58.184561+0000","172.21.15.12:0/441646478":"2024-09-17T14:12:58.184561+0000","172.21.15.12:6801/3186802043":"2024-09-17T14:12:58.184561+0000","172.21.15.12:6800/3186802043":"2024-09-17T14:12:58.184561+0000","172.21.15.12:0/714893902":"2024-09-17T14:12:58.184561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-mon[30490]: Standby manager daemon x restarted 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-mon[30490]: Standby manager daemon x started 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-mon[30490]: from='mgr.? 172.21.15.189:0/2886073600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T14:18:29.588 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:29.157+0000 7f8f79b9b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:18:29.589 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: [16/Sep/2024:14:18:29] ENGINE Bus STARTING 2024-09-16T14:18:29.589 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: CherryPy Checker: 2024-09-16T14:18:29.589 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: The Application mounted at '' has an empty config. 2024-09-16T14:18:29.589 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: 2024-09-16T14:18:29.589 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: [16/Sep/2024:14:18:29] ENGINE Serving on http://:::9283 2024-09-16T14:18:29.589 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:18:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x[32348]: [16/Sep/2024:14:18:29] ENGINE Bus STARTED 2024-09-16T14:18:29.736 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:29.560+0000 7fe3b2d1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:18:29.736 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:18:29] ENGINE Bus STARTING 2024-09-16T14:18:29.736 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: CherryPy Checker: 2024-09-16T14:18:29.736 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: The Application mounted at '' has an empty config. 2024-09-16T14:18:29.736 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:30.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:18:29] ENGINE Serving on http://:::9283 2024-09-16T14:18:30.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:18:29] ENGINE Bus STARTED 2024-09-16T14:18:30.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T14:18:30.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T14:18:30.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T14:18:30.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T14:18:30.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T14:18:30.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T14:18:30.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T14:18:30.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T14:18:30.405 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T14:18:30.405 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3706886765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:18:30.405 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: Active manager daemon y restarted 2024-09-16T14:18:30.405 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: Activating manager daemon y 2024-09-16T14:18:30.405 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T14:18:30.405 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: mgrmap e19: y(active, starting, since 0.00814361s), standbys: x 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:18:30.406 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: Manager daemon y is now available 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:18:30.407 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3706886765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: Active manager daemon y restarted 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: Activating manager daemon y 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: mgrmap e19: y(active, starting, since 0.00814361s), standbys: x 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:18:30.416 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:18:30.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: Manager daemon y is now available 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:18:30.418 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:30 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:18:30.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T14:18:30.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3706886765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T14:18:30.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: Active manager daemon y restarted 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: Activating manager daemon y 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: mgrmap e19: y(active, starting, since 0.00814361s), standbys: x 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T14:18:30.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: Manager daemon y is now available 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:18:30.590 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:30 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T14:18:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:31.748 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-mon[38187]: mgrmap e20: y(active, since 1.02433s), standbys: x 2024-09-16T14:18:31.748 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-mon[38187]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:31.749 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-mon[25528]: mgrmap e20: y(active, since 1.02433s), standbys: x 2024-09-16T14:18:31.749 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-mon[25528]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:31.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:31 smithi189 ceph-mon[30490]: mgrmap e20: y(active, since 1.02433s), standbys: x 2024-09-16T14:18:31.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:31 smithi189 ceph-mon[30490]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:32.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:31.867+0000 7fe307182640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:14:18:31] ENGINE Error in HTTPServer.serve 2024-09-16T14:18:32.027 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: Traceback (most recent call last): 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._connections.run(self.expiration_interval) 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._run(expiration_interval) 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: s = self.context.wrap_socket( 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:18:32.028 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: return self.sslsocket_class._create( 2024-09-16T14:18:32.029 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:18:32.029 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self.do_handshake() 2024-09-16T14:18:32.029 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:18:32.029 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: self._sslobj.do_handshake() 2024-09-16T14:18:32.029 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:18:32.029 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:31 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:18:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:32.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T14:18:32.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: [16/Sep/2024:14:18:31] ENGINE Bus STARTING 2024-09-16T14:18:32.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: [16/Sep/2024:14:18:31] ENGINE Serving on http://172.21.15.12:8765 2024-09-16T14:18:32.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: [16/Sep/2024:14:18:31] ENGINE Serving on https://172.21.15.12:7150 2024-09-16T14:18:32.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: [16/Sep/2024:14:18:31] ENGINE Bus STARTED 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: [16/Sep/2024:14:18:31] ENGINE Error in HTTPServer.serve 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: Traceback (most recent call last): 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: self._connections.run(self.expiration_interval) 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: self._run(expiration_interval) 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: s = self.context.wrap_socket( 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: return self.sslsocket_class._create( 2024-09-16T14:18:32.778 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.c.smithi012.stdout: self.do_handshake() 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.c.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.c.smithi012.stdout: self._sslobj.do_handshake() 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.c.smithi012.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: [16/Sep/2024:14:18:31] ENGINE Bus STARTING 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: [16/Sep/2024:14:18:31] ENGINE Serving on http://172.21.15.12:8765 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: [16/Sep/2024:14:18:31] ENGINE Serving on https://172.21.15.12:7150 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: [16/Sep/2024:14:18:31] ENGINE Bus STARTED 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: [16/Sep/2024:14:18:31] ENGINE Error in HTTPServer.serve 2024-09-16T14:18:32.779 INFO:journalctl@ceph.mon.a.smithi012.stdout: Traceback (most recent call last): 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._connections.run(self.expiration_interval) 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._run(expiration_interval) 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: s = self.context.wrap_socket( 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: return self.sslsocket_class._create( 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: self.do_handshake() 2024-09-16T14:18:32.780 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:18:32.781 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._sslobj.do_handshake() 2024-09-16T14:18:32.781 INFO:journalctl@ceph.mon.a.smithi012.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:18:32.781 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:32.781 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:32 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:32.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T14:18:32.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: [16/Sep/2024:14:18:31] ENGINE Bus STARTING 2024-09-16T14:18:32.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: [16/Sep/2024:14:18:31] ENGINE Serving on http://172.21.15.12:8765 2024-09-16T14:18:32.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: [16/Sep/2024:14:18:31] ENGINE Serving on https://172.21.15.12:7150 2024-09-16T14:18:32.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: [16/Sep/2024:14:18:31] ENGINE Bus STARTED 2024-09-16T14:18:32.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: [16/Sep/2024:14:18:31] ENGINE Error in HTTPServer.serve 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: Traceback (most recent call last): 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._connections.run(self.expiration_interval) 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._run(expiration_interval) 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: s = self.context.wrap_socket( 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: return self.sslsocket_class._create( 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: self.do_handshake() 2024-09-16T14:18:32.839 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T14:18:32.840 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._sslobj.do_handshake() 2024-09-16T14:18:32.840 INFO:journalctl@ceph.mon.b.smithi189.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T14:18:32.840 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:32.840 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:32 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:33.579 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:33.580 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:33.581 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:34.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:18:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-16T14:18:34.089 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:34 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-16T14:18:34.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.279 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.280 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.280 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:34.280 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-16T14:18:34.280 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:34 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:35.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:35 smithi189 ceph-mon[30490]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:35.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:35 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:35.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:35 smithi189 ceph-mon[30490]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T14:18:35.446 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:35 smithi012 ceph-mon[38187]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:35.446 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:35 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:35.446 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:35 smithi012 ceph-mon[38187]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T14:18:35.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:35 smithi012 ceph-mon[25528]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:35.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:35 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:35.447 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:35 smithi012 ceph-mon[25528]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T14:18:37.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:37 smithi189 ceph-mon[30490]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:37.372 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:37 smithi012 ceph-mon[38187]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:37.373 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:37 smithi012 ceph-mon[25528]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-09-16T14:18:39.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:39 smithi012 ceph-mon[38187]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T14:18:39.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:39 smithi012 ceph-mon[25528]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T14:18:39.278 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:39 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:18:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:18:39.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:39 smithi189 ceph-mon[30490]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T14:18:41.273 INFO:teuthology.orchestra.run.smithi012.stdout:163208757271 2024-09-16T14:18:41.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T14:18:41.298 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:41 smithi012 ceph-mon[38187]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T14:18:41.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:41 smithi012 ceph-mon[25528]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T14:18:41.337 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:41 smithi189 ceph-mon[30490]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T14:18:41.715 INFO:teuthology.orchestra.run.smithi012.stdout:107374182433 2024-09-16T14:18:41.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T14:18:43.276 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:43 smithi012 ceph-mon[38187]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T14:18:43.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:43 smithi012 ceph-mon[25528]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T14:18:43.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:43 smithi189 ceph-mon[30490]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T14:18:43.754 INFO:teuthology.orchestra.run.smithi012.stdout:236223201296 2024-09-16T14:18:43.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T14:18:44.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:43 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:18:44.857 INFO:teuthology.orchestra.run.smithi012.stdout:128849018911 2024-09-16T14:18:44.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T14:18:45.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:45 smithi012 ceph-mon[38187]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T14:18:45.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:45 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:45.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:45 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:18:45.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:45 smithi012 ceph-mon[25528]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T14:18:45.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:45 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:45.278 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:45 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:18:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:45 smithi189 ceph-mon[30490]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T14:18:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:45 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:45.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:45 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:18:45.406 INFO:teuthology.orchestra.run.smithi012.stdout:197568495636 2024-09-16T14:18:45.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T14:18:45.996 INFO:teuthology.orchestra.run.smithi012.stdout:77309411368 2024-09-16T14:18:45.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T14:18:46.768 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:47.051 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:47 smithi012 ceph-mon[38187]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T14:18:47.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:47 smithi012 ceph-mon[25528]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T14:18:47.338 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:47 smithi189 ceph-mon[30490]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T14:18:48.055 INFO:teuthology.orchestra.run.smithi012.stdout:55834574894 2024-09-16T14:18:48.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T14:18:48.284 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:48.284 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:48.780 INFO:teuthology.orchestra.run.smithi012.stdout:34359738419 2024-09-16T14:18:48.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T14:18:49.302 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:18:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:18:49.303 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-mon[38187]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T14:18:49.303 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:49.303 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:49.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-mon[25528]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T14:18:49.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:49.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:49 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:49.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:49 smithi189 ceph-mon[30490]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T14:18:49.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:49 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:49.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:49 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.366 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: Adjusting osd_memory_target on smithi012 to 3761M 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.367 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: Adjusting osd_memory_target on smithi012 to 3761M 2024-09-16T14:18:50.368 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.369 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:50 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T14:18:50.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: Adjusting osd_memory_target on smithi012 to 3761M 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.589 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:50 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:18:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:18:51.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[38187]: Deploying daemon alertmanager.a on smithi012 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:18:51.528 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:51 smithi012 ceph-mon[25528]: Deploying daemon alertmanager.a on smithi012 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Updating smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Updating smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T14:18:51.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:51 smithi189 ceph-mon[30490]: Deploying daemon alertmanager.a on smithi012 2024-09-16T14:18:53.504 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:53 smithi012 ceph-mon[38187]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:53.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:53 smithi012 ceph-mon[25528]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:53.587 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:53 smithi189 ceph-mon[30490]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:54.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:18:53 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:18:55.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:55 smithi012 ceph-mon[25528]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:55.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:55 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:55.495 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:55 smithi012 ceph-mon[38187]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:55.495 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:55 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:55.679 INFO:teuthology.orchestra.run.smithi012.stdout:197568495638 2024-09-16T14:18:55.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:55 smithi189 ceph-mon[30490]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:55.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:55 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:18:55.876 INFO:teuthology.orchestra.run.smithi012.stdout:107374182437 2024-09-16T14:18:56.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:56 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2776721532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T14:18:56.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:56 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3257144129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T14:18:56.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:56 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2776721532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T14:18:56.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:56 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3257144129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T14:18:56.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495636 got 197568495638 for osd.6 2024-09-16T14:18:56.662 DEBUG:teuthology.parallel:result is None 2024-09-16T14:18:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:56 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2776721532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T14:18:56.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:56 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3257144129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T14:18:57.495 INFO:teuthology.orchestra.run.smithi012.stdout:128849018913 2024-09-16T14:18:57.654 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:57 smithi012 ceph-mon[38187]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:57.654 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:57 smithi012 ceph-mon[25528]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:57.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:57 smithi189 ceph-mon[30490]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:18:57.926 INFO:teuthology.orchestra.run.smithi012.stdout:77309411370 2024-09-16T14:18:58.069 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:18:58.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:58 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1392286998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T14:18:58.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:58 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/4177714720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T14:18:58.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:58 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1392286998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T14:18:58.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:58 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/4177714720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T14:18:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:58 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1392286998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T14:18:58.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:58 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/4177714720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T14:18:59.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:18:59 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:18:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:18:59.375 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411370 for osd.2 2024-09-16T14:18:59.375 DEBUG:teuthology.parallel:result is None 2024-09-16T14:18:59.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:18:59 smithi012 ceph-mon[38187]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:59.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:18:59 smithi012 ceph-mon[25528]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:18:59.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:18:59 smithi189 ceph-mon[30490]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:00.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:00 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:00.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:00 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:00.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:00 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:00.868 INFO:teuthology.orchestra.run.smithi012.stdout:236223201299 2024-09-16T14:19:00.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182433 got 107374182437 for osd.3 2024-09-16T14:19:00.975 DEBUG:teuthology.parallel:result is None 2024-09-16T14:19:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:01.557 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018913 for osd.4 2024-09-16T14:19:01.557 DEBUG:teuthology.parallel:result is None 2024-09-16T14:19:01.659 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:01 smithi012 ceph-mon[38187]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T14:19:01.659 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:01 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2048367246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T14:19:01.659 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:01 smithi012 ceph-mon[25528]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T14:19:01.659 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:01 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2048367246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T14:19:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:01 smithi189 ceph-mon[30490]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T14:19:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:01 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2048367246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T14:19:02.110 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201296 got 236223201299 for osd.7 2024-09-16T14:19:02.110 DEBUG:teuthology.parallel:result is None 2024-09-16T14:19:03.180 INFO:teuthology.orchestra.run.smithi012.stdout:163208757276 2024-09-16T14:19:03.711 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:03 smithi012 ceph-mon[38187]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T14:19:03.711 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:03 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3638005147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T14:19:03.711 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:03 smithi012 ceph-mon[25528]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T14:19:03.711 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:03 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3638005147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T14:19:03.829 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:03 smithi189 ceph-mon[30490]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T14:19:03.829 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:03 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3638005147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T14:19:04.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757271 got 163208757276 for osd.5 2024-09-16T14:19:04.019 DEBUG:teuthology.parallel:result is None 2024-09-16T14:19:04.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:19:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:19:05.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:05 smithi012 ceph-mon[38187]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:05.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:05 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:05.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:05 smithi012 ceph-mon[25528]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:05.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:05 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:05.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:05 smithi189 ceph-mon[30490]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:05.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:05 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:06.017 INFO:teuthology.orchestra.run.smithi012.stdout:55834574897 2024-09-16T14:19:06.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:06 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/1613721797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T14:19:06.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:06 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/1613721797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T14:19:06.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574894 got 55834574897 for osd.1 2024-09-16T14:19:06.603 DEBUG:teuthology.parallel:result is None 2024-09-16T14:19:06.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:06 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/1613721797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T14:19:07.468 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:07 smithi012 ceph-mon[38187]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:07.468 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:07 smithi012 ceph-mon[25528]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:07.548 INFO:teuthology.orchestra.run.smithi012.stdout:34359738422 2024-09-16T14:19:07.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:07 smithi189 ceph-mon[30490]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:08.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738422 for osd.0 2024-09-16T14:19:08.357 DEBUG:teuthology.parallel:result is None 2024-09-16T14:19:08.357 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T14:19:08.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T14:19:08.690 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:08 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/3020941539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T14:19:08.691 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:08 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/3020941539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T14:19:08.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:08 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/3020941539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T14:19:09.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:09 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:19:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:19:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:09.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:09 smithi012 ceph-mon[38187]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:09.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:09 smithi012 ceph-mon[25528]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:09.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:09 smithi189 ceph-mon[30490]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:11.386 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:11 smithi012 systemd[1]: Starting Ceph alertmanager.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:11.674 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:11 smithi012 ceph-mon[38187]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:11.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:11 smithi012 ceph-mon[25528]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:11.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:11 smithi189 ceph-mon[30490]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:11.973 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:11 smithi012 podman[78548]: 2024-09-16 14:19:11.668662017 +0000 UTC m=+0.195464024 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-16T14:19:11.973 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:11 smithi012 podman[78548]: 2024-09-16 14:19:11.787680784 +0000 UTC m=+0.314482791 volume create 90ad9464ad4d3710d01e75fcbbeff74831cd07c5319429dfa54ae6a3afd0e4b3 2024-09-16T14:19:12.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:19:12.148 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T14:19:12.277 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:12 smithi012 podman[78548]: 2024-09-16 14:19:12.097281101 +0000 UTC m=+0.624083105 container create c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T14:19:12.999 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T14:19:11.586613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1023,"num_read_kb":866,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1771764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":252864,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":749675584,"statfs":{"total":767926730752,"available":767667798016,"internally_reserved":0,"allocated":8232960,"data_stored":4472899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":250596921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002635"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771329+0000","last_change":"2024-09-16T14:17:57.712867+0000","last_active":"2024-09-16T14:18:22.771329+0000","last_peered":"2024-09-16T14:18:22.771329+0000","last_clean":"2024-09-16T14:18:22.771329+0000","last_became_active":"2024-09-16T14:17:57.712556+0000","last_became_peered":"2024-09-16T14:17:57.712556+0000","last_unstale":"2024-09-16T14:18:22.771329+0000","last_undegraded":"2024-09-16T14:18:22.771329+0000","last_fullsized":"2024-09-16T14:18:22.771329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:58.734845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.778540+0000","last_change":"2024-09-16T14:17:51.397770+0000","last_active":"2024-09-16T14:18:22.778540+0000","last_peered":"2024-09-16T14:18:22.778540+0000","last_clean":"2024-09-16T14:18:22.778540+0000","last_became_active":"2024-09-16T14:17:51.397492+0000","last_became_peered":"2024-09-16T14:17:51.397492+0000","last_unstale":"2024-09-16T14:18:22.778540+0000","last_undegraded":"2024-09-16T14:18:22.778540+0000","last_fullsized":"2024-09-16T14:18:22.778540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:41.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771541+0000","last_change":"2024-09-16T14:17:53.473856+0000","last_active":"2024-09-16T14:18:22.771541+0000","last_peered":"2024-09-16T14:18:22.771541+0000","last_clean":"2024-09-16T14:18:22.771541+0000","last_became_active":"2024-09-16T14:17:53.473534+0000","last_became_peered":"2024-09-16T14:17:53.473534+0000","last_unstale":"2024-09-16T14:18:22.771541+0000","last_undegraded":"2024-09-16T14:18:22.771541+0000","last_fullsized":"2024-09-16T14:18:22.771541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:38:18.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771346+0000","last_change":"2024-09-16T14:17:55.493162+0000","last_active":"2024-09-16T14:18:22.771346+0000","last_peered":"2024-09-16T14:18:22.771346+0000","last_clean":"2024-09-16T14:18:22.771346+0000","last_became_active":"2024-09-16T14:17:55.492888+0000","last_became_peered":"2024-09-16T14:17:55.492888+0000","last_unstale":"2024-09-16T14:18:22.771346+0000","last_undegraded":"2024-09-16T14:18:22.771346+0000","last_fullsized":"2024-09-16T14:18:22.771346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:28.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772874+0000","last_change":"2024-09-16T14:17:51.403637+0000","last_active":"2024-09-16T14:18:22.772874+0000","last_peered":"2024-09-16T14:18:22.772874+0000","last_clean":"2024-09-16T14:18:22.772874+0000","last_became_active":"2024-09-16T14:17:51.403353+0000","last_became_peered":"2024-09-16T14:17:51.403353+0000","last_unstale":"2024-09-16T14:18:22.772874+0000","last_undegraded":"2024-09-16T14:18:22.772874+0000","last_fullsized":"2024-09-16T14:18:22.772874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:59.668377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771354+0000","last_change":"2024-09-16T14:17:53.484168+0000","last_active":"2024-09-16T14:18:22.771354+0000","last_peered":"2024-09-16T14:18:22.771354+0000","last_clean":"2024-09-16T14:18:22.771354+0000","last_became_active":"2024-09-16T14:17:53.483661+0000","last_became_peered":"2024-09-16T14:17:53.483661+0000","last_unstale":"2024-09-16T14:18:22.771354+0000","last_undegraded":"2024-09-16T14:18:22.771354+0000","last_fullsized":"2024-09-16T14:18:22.771354+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:46.765389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771625+0000","last_change":"2024-09-16T14:17:55.492869+0000","last_active":"2024-09-16T14:18:22.771625+0000","last_peered":"2024-09-16T14:18:22.771625+0000","last_clean":"2024-09-16T14:18:22.771625+0000","last_became_active":"2024-09-16T14:17:55.492586+0000","last_became_peered":"2024-09-16T14:17:55.492586+0000","last_unstale":"2024-09-16T14:18:22.771625+0000","last_undegraded":"2024-09-16T14:18:22.771625+0000","last_fullsized":"2024-09-16T14:18:22.771625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:45.289151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771646+0000","last_change":"2024-09-16T14:17:57.512096+0000","last_active":"2024-09-16T14:18:22.771646+0000","last_peered":"2024-09-16T14:18:22.771646+0000","last_clean":"2024-09-16T14:18:22.771646+0000","last_became_active":"2024-09-16T14:17:57.511727+0000","last_became_peered":"2024-09-16T14:17:57.511727+0000","last_unstale":"2024-09-16T14:18:22.771646+0000","last_undegraded":"2024-09-16T14:18:22.771646+0000","last_fullsized":"2024-09-16T14:18:22.771646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:30.564310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928687+0000","last_change":"2024-09-16T14:17:51.403697+0000","last_active":"2024-09-16T14:18:22.928687+0000","last_peered":"2024-09-16T14:18:22.928687+0000","last_clean":"2024-09-16T14:18:22.928687+0000","last_became_active":"2024-09-16T14:17:51.403443+0000","last_became_peered":"2024-09-16T14:17:51.403443+0000","last_unstale":"2024-09-16T14:18:22.928687+0000","last_undegraded":"2024-09-16T14:18:22.928687+0000","last_fullsized":"2024-09-16T14:18:22.928687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:26.861141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771541+0000","last_change":"2024-09-16T14:17:53.484042+0000","last_active":"2024-09-16T14:18:22.771541+0000","last_peered":"2024-09-16T14:18:22.771541+0000","last_clean":"2024-09-16T14:18:22.771541+0000","last_became_active":"2024-09-16T14:17:53.483630+0000","last_became_peered":"2024-09-16T14:17:53.483630+0000","last_unstale":"2024-09-16T14:18:22.771541+0000","last_undegraded":"2024-09-16T14:18:22.771541+0000","last_fullsized":"2024-09-16T14:18:22.771541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:21.900570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928225+0000","last_change":"2024-09-16T14:17:55.497028+0000","last_active":"2024-09-16T14:18:22.928225+0000","last_peered":"2024-09-16T14:18:22.928225+0000","last_clean":"2024-09-16T14:18:22.928225+0000","last_became_active":"2024-09-16T14:17:55.496751+0000","last_became_peered":"2024-09-16T14:17:55.496751+0000","last_unstale":"2024-09-16T14:18:22.928225+0000","last_undegraded":"2024-09-16T14:18:22.928225+0000","last_fullsized":"2024-09-16T14:18:22.928225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:59.979638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776349+0000","last_change":"2024-09-16T14:17:57.510856+0000","last_active":"2024-09-16T14:18:22.776349+0000","last_peered":"2024-09-16T14:18:22.776349+0000","last_clean":"2024-09-16T14:18:22.776349+0000","last_became_active":"2024-09-16T14:17:57.510529+0000","last_became_peered":"2024-09-16T14:17:57.510529+0000","last_unstale":"2024-09-16T14:18:22.776349+0000","last_undegraded":"2024-09-16T14:18:22.776349+0000","last_fullsized":"2024-09-16T14:18:22.776349+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:39.547665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928574+0000","last_change":"2024-09-16T14:17:51.400901+0000","last_active":"2024-09-16T14:18:22.928574+0000","last_peered":"2024-09-16T14:18:22.928574+0000","last_clean":"2024-09-16T14:18:22.928574+0000","last_became_active":"2024-09-16T14:17:51.400635+0000","last_became_peered":"2024-09-16T14:17:51.400635+0000","last_unstale":"2024-09-16T14:18:22.928574+0000","last_undegraded":"2024-09-16T14:18:22.928574+0000","last_fullsized":"2024-09-16T14:18:22.928574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:34.534106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771908+0000","last_change":"2024-09-16T14:17:53.471486+0000","last_active":"2024-09-16T14:18:22.771908+0000","last_peered":"2024-09-16T14:18:22.771908+0000","last_clean":"2024-09-16T14:18:22.771908+0000","last_became_active":"2024-09-16T14:17:53.470940+0000","last_became_peered":"2024-09-16T14:17:53.470940+0000","last_unstale":"2024-09-16T14:18:22.771908+0000","last_undegraded":"2024-09-16T14:18:22.771908+0000","last_fullsized":"2024-09-16T14:18:22.771908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:39.782219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776605+0000","last_change":"2024-09-16T14:17:55.494933+0000","last_active":"2024-09-16T14:18:22.776605+0000","last_peered":"2024-09-16T14:18:22.776605+0000","last_clean":"2024-09-16T14:18:22.776605+0000","last_became_active":"2024-09-16T14:17:55.494614+0000","last_became_peered":"2024-09-16T14:17:55.494614+0000","last_unstale":"2024-09-16T14:18:22.776605+0000","last_undegraded":"2024-09-16T14:18:22.776605+0000","last_fullsized":"2024-09-16T14:18:22.776605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:14.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776538+0000","last_change":"2024-09-16T14:17:57.519096+0000","last_active":"2024-09-16T14:18:22.776538+0000","last_peered":"2024-09-16T14:18:22.776538+0000","last_clean":"2024-09-16T14:18:22.776538+0000","last_became_active":"2024-09-16T14:17:57.518424+0000","last_became_peered":"2024-09-16T14:17:57.518424+0000","last_unstale":"2024-09-16T14:18:22.776538+0000","last_undegraded":"2024-09-16T14:18:22.776538+0000","last_fullsized":"2024-09-16T14:18:22.776538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:26.662229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771913+0000","last_change":"2024-09-16T14:17:53.480478+0000","last_active":"2024-09-16T14:18:22.771913+0000","last_peered":"2024-09-16T14:18:22.771913+0000","last_clean":"2024-09-16T14:18:22.771913+0000","last_became_active":"2024-09-16T14:17:53.480159+0000","last_became_peered":"2024-09-16T14:17:53.480159+0000","last_unstale":"2024-09-16T14:18:22.771913+0000","last_undegraded":"2024-09-16T14:18:22.771913+0000","last_fullsized":"2024-09-16T14:18:22.771913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:50:54.318096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928713+0000","last_change":"2024-09-16T14:17:51.394867+0000","last_active":"2024-09-16T14:18:22.928713+0000","last_peered":"2024-09-16T14:18:22.928713+0000","last_clean":"2024-09-16T14:18:22.928713+0000","last_became_active":"2024-09-16T14:17:51.394341+0000","last_became_peered":"2024-09-16T14:17:51.394341+0000","last_unstale":"2024-09-16T14:18:22.928713+0000","last_undegraded":"2024-09-16T14:18:22.928713+0000","last_fullsized":"2024-09-16T14:18:22.928713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:41.051615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771919+0000","last_change":"2024-09-16T14:17:55.498471+0000","last_active":"2024-09-16T14:18:22.771919+0000","last_peered":"2024-09-16T14:18:22.771919+0000","last_clean":"2024-09-16T14:18:22.771919+0000","last_became_active":"2024-09-16T14:17:55.496725+0000","last_became_peered":"2024-09-16T14:17:55.496725+0000","last_unstale":"2024-09-16T14:18:22.771919+0000","last_undegraded":"2024-09-16T14:18:22.771919+0000","last_fullsized":"2024-09-16T14:18:22.771919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:28.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777116+0000","last_change":"2024-09-16T14:17:57.512976+0000","last_active":"2024-09-16T14:18:22.777116+0000","last_peered":"2024-09-16T14:18:22.777116+0000","last_clean":"2024-09-16T14:18:22.777116+0000","last_became_active":"2024-09-16T14:17:57.512585+0000","last_became_peered":"2024-09-16T14:17:57.512585+0000","last_unstale":"2024-09-16T14:18:22.777116+0000","last_undegraded":"2024-09-16T14:18:22.777116+0000","last_fullsized":"2024-09-16T14:18:22.777116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:09:38.889476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771678+0000","last_change":"2024-09-16T14:17:53.470204+0000","last_active":"2024-09-16T14:18:22.771678+0000","last_peered":"2024-09-16T14:18:22.771678+0000","last_clean":"2024-09-16T14:18:22.771678+0000","last_became_active":"2024-09-16T14:17:53.469881+0000","last_became_peered":"2024-09-16T14:17:53.469881+0000","last_unstale":"2024-09-16T14:18:22.771678+0000","last_undegraded":"2024-09-16T14:18:22.771678+0000","last_fullsized":"2024-09-16T14:18:22.771678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:45.758115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772303+0000","last_change":"2024-09-16T14:17:51.399418+0000","last_active":"2024-09-16T14:18:22.772303+0000","last_peered":"2024-09-16T14:18:22.772303+0000","last_clean":"2024-09-16T14:18:22.772303+0000","last_became_active":"2024-09-16T14:17:51.399139+0000","last_became_peered":"2024-09-16T14:17:51.399139+0000","last_unstale":"2024-09-16T14:18:22.772303+0000","last_undegraded":"2024-09-16T14:18:22.772303+0000","last_fullsized":"2024-09-16T14:18:22.772303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:29.393614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511431+0000","last_change":"2024-09-16T14:17:55.498938+0000","last_active":"2024-09-16T14:19:04.511431+0000","last_peered":"2024-09-16T14:19:04.511431+0000","last_clean":"2024-09-16T14:19:04.511431+0000","last_became_active":"2024-09-16T14:17:55.498521+0000","last_became_peered":"2024-09-16T14:17:55.498521+0000","last_unstale":"2024-09-16T14:19:04.511431+0000","last_undegraded":"2024-09-16T14:19:04.511431+0000","last_fullsized":"2024-09-16T14:19:04.511431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:21.593867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771930+0000","last_change":"2024-09-16T14:17:57.516399+0000","last_active":"2024-09-16T14:18:22.771930+0000","last_peered":"2024-09-16T14:18:22.771930+0000","last_clean":"2024-09-16T14:18:22.771930+0000","last_became_active":"2024-09-16T14:17:57.516111+0000","last_became_peered":"2024-09-16T14:17:57.516111+0000","last_unstale":"2024-09-16T14:18:22.771930+0000","last_undegraded":"2024-09-16T14:18:22.771930+0000","last_fullsized":"2024-09-16T14:18:22.771930+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:53.039153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772678+0000","last_change":"2024-09-16T14:17:53.481417+0000","last_active":"2024-09-16T14:18:22.772678+0000","last_peered":"2024-09-16T14:18:22.772678+0000","last_clean":"2024-09-16T14:18:22.772678+0000","last_became_active":"2024-09-16T14:17:53.479685+0000","last_became_peered":"2024-09-16T14:17:53.479685+0000","last_unstale":"2024-09-16T14:18:22.772678+0000","last_undegraded":"2024-09-16T14:18:22.772678+0000","last_fullsized":"2024-09-16T14:18:22.772678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:10.649254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771771+0000","last_change":"2024-09-16T14:17:51.395131+0000","last_active":"2024-09-16T14:18:22.771771+0000","last_peered":"2024-09-16T14:18:22.771771+0000","last_clean":"2024-09-16T14:18:22.771771+0000","last_became_active":"2024-09-16T14:17:51.394857+0000","last_became_peered":"2024-09-16T14:17:51.394857+0000","last_unstale":"2024-09-16T14:18:22.771771+0000","last_undegraded":"2024-09-16T14:18:22.771771+0000","last_fullsized":"2024-09-16T14:18:22.771771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:29.471562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511587+0000","last_change":"2024-09-16T14:17:55.490831+0000","last_active":"2024-09-16T14:19:04.511587+0000","last_peered":"2024-09-16T14:19:04.511587+0000","last_clean":"2024-09-16T14:19:04.511587+0000","last_became_active":"2024-09-16T14:17:55.490362+0000","last_became_peered":"2024-09-16T14:17:55.490362+0000","last_unstale":"2024-09-16T14:19:04.511587+0000","last_undegraded":"2024-09-16T14:19:04.511587+0000","last_fullsized":"2024-09-16T14:19:04.511587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:31.796055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776728+0000","last_change":"2024-09-16T14:17:57.515312+0000","last_active":"2024-09-16T14:18:22.776728+0000","last_peered":"2024-09-16T14:18:22.776728+0000","last_clean":"2024-09-16T14:18:22.776728+0000","last_became_active":"2024-09-16T14:17:57.515044+0000","last_became_peered":"2024-09-16T14:17:57.515044+0000","last_unstale":"2024-09-16T14:18:22.776728+0000","last_undegraded":"2024-09-16T14:18:22.776728+0000","last_fullsized":"2024-09-16T14:18:22.776728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:33.969061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772106+0000","last_change":"2024-09-16T14:17:53.477474+0000","last_active":"2024-09-16T14:18:22.772106+0000","last_peered":"2024-09-16T14:18:22.772106+0000","last_clean":"2024-09-16T14:18:22.772106+0000","last_became_active":"2024-09-16T14:17:53.477144+0000","last_became_peered":"2024-09-16T14:17:53.477144+0000","last_unstale":"2024-09-16T14:18:22.772106+0000","last_undegraded":"2024-09-16T14:18:22.772106+0000","last_fullsized":"2024-09-16T14:18:22.772106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:59.759550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928161+0000","last_change":"2024-09-16T14:17:51.399942+0000","last_active":"2024-09-16T14:18:22.928161+0000","last_peered":"2024-09-16T14:18:22.928161+0000","last_clean":"2024-09-16T14:18:22.928161+0000","last_became_active":"2024-09-16T14:17:51.399586+0000","last_became_peered":"2024-09-16T14:17:51.399586+0000","last_unstale":"2024-09-16T14:18:22.928161+0000","last_undegraded":"2024-09-16T14:18:22.928161+0000","last_fullsized":"2024-09-16T14:18:22.928161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:16.495921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777084+0000","last_change":"2024-09-16T14:17:55.496534+0000","last_active":"2024-09-16T14:18:22.777084+0000","last_peered":"2024-09-16T14:18:22.777084+0000","last_clean":"2024-09-16T14:18:22.777084+0000","last_became_active":"2024-09-16T14:17:55.496189+0000","last_became_peered":"2024-09-16T14:17:55.496189+0000","last_unstale":"2024-09-16T14:18:22.777084+0000","last_undegraded":"2024-09-16T14:18:22.777084+0000","last_fullsized":"2024-09-16T14:18:22.777084+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:52.395816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771669+0000","last_change":"2024-09-16T14:17:57.712103+0000","last_active":"2024-09-16T14:18:22.771669+0000","last_peered":"2024-09-16T14:18:22.771669+0000","last_clean":"2024-09-16T14:18:22.771669+0000","last_became_active":"2024-09-16T14:17:57.711600+0000","last_became_peered":"2024-09-16T14:17:57.711600+0000","last_unstale":"2024-09-16T14:18:22.771669+0000","last_undegraded":"2024-09-16T14:18:22.771669+0000","last_fullsized":"2024-09-16T14:18:22.771669+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:37:34.394514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771646+0000","last_change":"2024-09-16T14:17:53.471476+0000","last_active":"2024-09-16T14:18:22.771646+0000","last_peered":"2024-09-16T14:18:22.771646+0000","last_clean":"2024-09-16T14:18:22.771646+0000","last_became_active":"2024-09-16T14:17:53.471147+0000","last_became_peered":"2024-09-16T14:17:53.471147+0000","last_unstale":"2024-09-16T14:18:22.771646+0000","last_undegraded":"2024-09-16T14:18:22.771646+0000","last_fullsized":"2024-09-16T14:18:22.771646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:29.051796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928673+0000","last_change":"2024-09-16T14:17:51.400293+0000","last_active":"2024-09-16T14:18:22.928673+0000","last_peered":"2024-09-16T14:18:22.928673+0000","last_clean":"2024-09-16T14:18:22.928673+0000","last_became_active":"2024-09-16T14:17:51.400019+0000","last_became_peered":"2024-09-16T14:17:51.400019+0000","last_unstale":"2024-09-16T14:18:22.928673+0000","last_undegraded":"2024-09-16T14:18:22.928673+0000","last_fullsized":"2024-09-16T14:18:22.928673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:37.720347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511428+0000","last_change":"2024-09-16T14:17:55.489742+0000","last_active":"2024-09-16T14:19:04.511428+0000","last_peered":"2024-09-16T14:19:04.511428+0000","last_clean":"2024-09-16T14:19:04.511428+0000","last_became_active":"2024-09-16T14:17:55.489409+0000","last_became_peered":"2024-09-16T14:17:55.489409+0000","last_unstale":"2024-09-16T14:19:04.511428+0000","last_undegraded":"2024-09-16T14:19:04.511428+0000","last_fullsized":"2024-09-16T14:19:04.511428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:14.493465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928431+0000","last_change":"2024-09-16T14:17:57.711640+0000","last_active":"2024-09-16T14:18:22.928431+0000","last_peered":"2024-09-16T14:18:22.928431+0000","last_clean":"2024-09-16T14:18:22.928431+0000","last_became_active":"2024-09-16T14:17:57.711354+0000","last_became_peered":"2024-09-16T14:17:57.711354+0000","last_unstale":"2024-09-16T14:18:22.928431+0000","last_undegraded":"2024-09-16T14:18:22.928431+0000","last_fullsized":"2024-09-16T14:18:22.928431+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:11:03.914929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771141+0000","last_change":"2024-09-16T14:17:53.850576+0000","last_active":"2024-09-16T14:18:22.771141+0000","last_peered":"2024-09-16T14:18:22.771141+0000","last_clean":"2024-09-16T14:18:22.771141+0000","last_became_active":"2024-09-16T14:17:53.850315+0000","last_became_peered":"2024-09-16T14:17:53.850315+0000","last_unstale":"2024-09-16T14:18:22.771141+0000","last_undegraded":"2024-09-16T14:18:22.771141+0000","last_fullsized":"2024-09-16T14:18:22.771141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:51.795184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.773078+0000","last_change":"2024-09-16T14:17:51.393041+0000","last_active":"2024-09-16T14:18:22.773078+0000","last_peered":"2024-09-16T14:18:22.773078+0000","last_clean":"2024-09-16T14:18:22.773078+0000","last_became_active":"2024-09-16T14:17:51.392504+0000","last_became_peered":"2024-09-16T14:17:51.392504+0000","last_unstale":"2024-09-16T14:18:22.773078+0000","last_undegraded":"2024-09-16T14:18:22.773078+0000","last_fullsized":"2024-09-16T14:18:22.773078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:24.307139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771409+0000","last_change":"2024-09-16T14:17:55.492408+0000","last_active":"2024-09-16T14:18:22.771409+0000","last_peered":"2024-09-16T14:18:22.771409+0000","last_clean":"2024-09-16T14:18:22.771409+0000","last_became_active":"2024-09-16T14:17:55.492127+0000","last_became_peered":"2024-09-16T14:17:55.492127+0000","last_unstale":"2024-09-16T14:18:22.771409+0000","last_undegraded":"2024-09-16T14:18:22.771409+0000","last_fullsized":"2024-09-16T14:18:22.771409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:29.480789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771826+0000","last_change":"2024-09-16T14:17:57.516429+0000","last_active":"2024-09-16T14:18:22.771826+0000","last_peered":"2024-09-16T14:18:22.771826+0000","last_clean":"2024-09-16T14:18:22.771826+0000","last_became_active":"2024-09-16T14:17:57.516125+0000","last_became_peered":"2024-09-16T14:17:57.516125+0000","last_unstale":"2024-09-16T14:18:22.771826+0000","last_undegraded":"2024-09-16T14:18:22.771826+0000","last_fullsized":"2024-09-16T14:18:22.771826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:01:59.715466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":55,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.778943+0000","last_change":"2024-09-16T14:17:53.850520+0000","last_active":"2024-09-16T14:18:22.778943+0000","last_peered":"2024-09-16T14:18:22.778943+0000","last_clean":"2024-09-16T14:18:22.778943+0000","last_became_active":"2024-09-16T14:17:53.850092+0000","last_became_peered":"2024-09-16T14:17:53.850092+0000","last_unstale":"2024-09-16T14:18:22.778943+0000","last_undegraded":"2024-09-16T14:18:22.778943+0000","last_fullsized":"2024-09-16T14:18:22.778943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:04.790941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928768+0000","last_change":"2024-09-16T14:17:51.403722+0000","last_active":"2024-09-16T14:18:22.928768+0000","last_peered":"2024-09-16T14:18:22.928768+0000","last_clean":"2024-09-16T14:18:22.928768+0000","last_became_active":"2024-09-16T14:17:51.403464+0000","last_became_peered":"2024-09-16T14:17:51.403464+0000","last_unstale":"2024-09-16T14:18:22.928768+0000","last_undegraded":"2024-09-16T14:18:22.928768+0000","last_fullsized":"2024-09-16T14:18:22.928768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:09.494838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928087+0000","last_change":"2024-09-16T14:17:55.495768+0000","last_active":"2024-09-16T14:18:22.928087+0000","last_peered":"2024-09-16T14:18:22.928087+0000","last_clean":"2024-09-16T14:18:22.928087+0000","last_became_active":"2024-09-16T14:17:55.495463+0000","last_became_peered":"2024-09-16T14:17:55.495463+0000","last_unstale":"2024-09-16T14:18:22.928087+0000","last_undegraded":"2024-09-16T14:18:22.928087+0000","last_fullsized":"2024-09-16T14:18:22.928087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:17.943562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771079+0000","last_change":"2024-09-16T14:17:57.711945+0000","last_active":"2024-09-16T14:18:22.771079+0000","last_peered":"2024-09-16T14:18:22.771079+0000","last_clean":"2024-09-16T14:18:22.771079+0000","last_became_active":"2024-09-16T14:17:57.711588+0000","last_became_peered":"2024-09-16T14:17:57.711588+0000","last_unstale":"2024-09-16T14:18:22.771079+0000","last_undegraded":"2024-09-16T14:18:22.771079+0000","last_fullsized":"2024-09-16T14:18:22.771079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:52.132678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772157+0000","last_change":"2024-09-16T14:17:53.481354+0000","last_active":"2024-09-16T14:18:22.772157+0000","last_peered":"2024-09-16T14:18:22.772157+0000","last_clean":"2024-09-16T14:18:22.772157+0000","last_became_active":"2024-09-16T14:17:53.480091+0000","last_became_peered":"2024-09-16T14:17:53.480091+0000","last_unstale":"2024-09-16T14:18:22.772157+0000","last_undegraded":"2024-09-16T14:18:22.772157+0000","last_fullsized":"2024-09-16T14:18:22.772157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:44.963701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.773322+0000","last_change":"2024-09-16T14:17:51.396471+0000","last_active":"2024-09-16T14:18:22.773322+0000","last_peered":"2024-09-16T14:18:22.773322+0000","last_clean":"2024-09-16T14:18:22.773322+0000","last_became_active":"2024-09-16T14:17:51.396185+0000","last_became_peered":"2024-09-16T14:17:51.396185+0000","last_unstale":"2024-09-16T14:18:22.773322+0000","last_undegraded":"2024-09-16T14:18:22.773322+0000","last_fullsized":"2024-09-16T14:18:22.773322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:55.971443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511408+0000","last_change":"2024-09-16T14:17:55.502290+0000","last_active":"2024-09-16T14:19:04.511408+0000","last_peered":"2024-09-16T14:19:04.511408+0000","last_clean":"2024-09-16T14:19:04.511408+0000","last_became_active":"2024-09-16T14:17:55.501860+0000","last_became_peered":"2024-09-16T14:17:55.501860+0000","last_unstale":"2024-09-16T14:19:04.511408+0000","last_undegraded":"2024-09-16T14:19:04.511408+0000","last_fullsized":"2024-09-16T14:19:04.511408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:31.686145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776386+0000","last_change":"2024-09-16T14:17:57.518096+0000","last_active":"2024-09-16T14:18:22.776386+0000","last_peered":"2024-09-16T14:18:22.776386+0000","last_clean":"2024-09-16T14:18:22.776386+0000","last_became_active":"2024-09-16T14:17:57.517013+0000","last_became_peered":"2024-09-16T14:17:57.517013+0000","last_unstale":"2024-09-16T14:18:22.776386+0000","last_undegraded":"2024-09-16T14:18:22.776386+0000","last_fullsized":"2024-09-16T14:18:22.776386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:08.512697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":83,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511421+0000","last_change":"2024-09-16T14:17:53.481524+0000","last_active":"2024-09-16T14:19:04.511421+0000","last_peered":"2024-09-16T14:19:04.511421+0000","last_clean":"2024-09-16T14:19:04.511421+0000","last_became_active":"2024-09-16T14:17:53.481229+0000","last_became_peered":"2024-09-16T14:17:53.481229+0000","last_unstale":"2024-09-16T14:19:04.511421+0000","last_undegraded":"2024-09-16T14:19:04.511421+0000","last_fullsized":"2024-09-16T14:19:04.511421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:04.183728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776620+0000","last_change":"2024-09-16T14:17:51.393181+0000","last_active":"2024-09-16T14:18:22.776620+0000","last_peered":"2024-09-16T14:18:22.776620+0000","last_clean":"2024-09-16T14:18:22.776620+0000","last_became_active":"2024-09-16T14:17:51.392893+0000","last_became_peered":"2024-09-16T14:17:51.392893+0000","last_unstale":"2024-09-16T14:18:22.776620+0000","last_undegraded":"2024-09-16T14:18:22.776620+0000","last_fullsized":"2024-09-16T14:18:22.776620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777870+0000","last_change":"2024-09-16T14:17:55.496800+0000","last_active":"2024-09-16T14:18:22.777870+0000","last_peered":"2024-09-16T14:18:22.777870+0000","last_clean":"2024-09-16T14:18:22.777870+0000","last_became_active":"2024-09-16T14:17:55.496100+0000","last_became_peered":"2024-09-16T14:17:55.496100+0000","last_unstale":"2024-09-16T14:18:22.777870+0000","last_undegraded":"2024-09-16T14:18:22.777870+0000","last_fullsized":"2024-09-16T14:18:22.777870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:34:39.524984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772557+0000","last_change":"2024-09-16T14:17:57.515785+0000","last_active":"2024-09-16T14:18:22.772557+0000","last_peered":"2024-09-16T14:18:22.772557+0000","last_clean":"2024-09-16T14:18:22.772557+0000","last_became_active":"2024-09-16T14:17:57.515035+0000","last_became_peered":"2024-09-16T14:17:57.515035+0000","last_unstale":"2024-09-16T14:18:22.772557+0000","last_undegraded":"2024-09-16T14:18:22.772557+0000","last_fullsized":"2024-09-16T14:18:22.772557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:28.487955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:23.128445+0000","last_change":"2024-09-16T14:18:22.787419+0000","last_active":"2024-09-16T14:18:23.128445+0000","last_peered":"2024-09-16T14:18:23.128445+0000","last_clean":"2024-09-16T14:18:23.128445+0000","last_became_active":"2024-09-16T14:18:22.779558+0000","last_became_peered":"2024-09-16T14:18:22.779558+0000","last_unstale":"2024-09-16T14:18:23.128445+0000","last_undegraded":"2024-09-16T14:18:23.128445+0000","last_fullsized":"2024-09-16T14:18:23.128445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:58.019132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":101,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:08.885266+0000","last_change":"2024-09-16T14:17:55.483857+0000","last_active":"2024-09-16T14:19:08.885266+0000","last_peered":"2024-09-16T14:19:08.885266+0000","last_clean":"2024-09-16T14:19:08.885266+0000","last_became_active":"2024-09-16T14:17:51.399163+0000","last_became_peered":"2024-09-16T14:17:51.399163+0000","last_unstale":"2024-09-16T14:19:08.885266+0000","last_undegraded":"2024-09-16T14:19:08.885266+0000","last_fullsized":"2024-09-16T14:19:08.885266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:44.432547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035163099999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":72,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"61'1","reported_seq":26,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776000+0000","last_change":"2024-09-16T14:17:51.404809+0000","last_active":"2024-09-16T14:18:22.776000+0000","last_peered":"2024-09-16T14:18:22.776000+0000","last_clean":"2024-09-16T14:18:22.776000+0000","last_became_active":"2024-09-16T14:17:51.404285+0000","last_became_peered":"2024-09-16T14:17:51.404285+0000","last_unstale":"2024-09-16T14:18:22.776000+0000","last_undegraded":"2024-09-16T14:18:22.776000+0000","last_fullsized":"2024-09-16T14:18:22.776000+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:12:30.629923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776733+0000","last_change":"2024-09-16T14:17:55.499744+0000","last_active":"2024-09-16T14:18:22.776733+0000","last_peered":"2024-09-16T14:18:22.776733+0000","last_clean":"2024-09-16T14:18:22.776733+0000","last_became_active":"2024-09-16T14:17:55.499432+0000","last_became_peered":"2024-09-16T14:17:55.499432+0000","last_unstale":"2024-09-16T14:18:22.776733+0000","last_undegraded":"2024-09-16T14:18:22.776733+0000","last_fullsized":"2024-09-16T14:18:22.776733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:24.141736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"68'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928279+0000","last_change":"2024-09-16T14:17:57.711153+0000","last_active":"2024-09-16T14:18:22.928279+0000","last_peered":"2024-09-16T14:18:22.928279+0000","last_clean":"2024-09-16T14:18:22.928279+0000","last_became_active":"2024-09-16T14:17:57.710876+0000","last_became_peered":"2024-09-16T14:17:57.710876+0000","last_unstale":"2024-09-16T14:18:22.928279+0000","last_undegraded":"2024-09-16T14:18:22.928279+0000","last_fullsized":"2024-09-16T14:18:22.928279+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:22.083959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":37,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776755+0000","last_change":"2024-09-16T14:17:53.477253+0000","last_active":"2024-09-16T14:18:22.776755+0000","last_peered":"2024-09-16T14:18:22.776755+0000","last_clean":"2024-09-16T14:18:22.776755+0000","last_became_active":"2024-09-16T14:17:53.476948+0000","last_became_peered":"2024-09-16T14:17:53.476948+0000","last_unstale":"2024-09-16T14:18:22.776755+0000","last_undegraded":"2024-09-16T14:18:22.776755+0000","last_fullsized":"2024-09-16T14:18:22.776755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:29.443726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":27,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777589+0000","last_change":"2024-09-16T14:17:55.488175+0000","last_active":"2024-09-16T14:18:22.777589+0000","last_peered":"2024-09-16T14:18:22.777589+0000","last_clean":"2024-09-16T14:18:22.777589+0000","last_became_active":"2024-09-16T14:17:51.396572+0000","last_became_peered":"2024-09-16T14:17:51.396572+0000","last_unstale":"2024-09-16T14:18:22.777589+0000","last_undegraded":"2024-09-16T14:18:22.777589+0000","last_fullsized":"2024-09-16T14:18:22.777589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:50:22.697464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049759299999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928238+0000","last_change":"2024-09-16T14:17:51.400300+0000","last_active":"2024-09-16T14:18:22.928238+0000","last_peered":"2024-09-16T14:18:22.928238+0000","last_clean":"2024-09-16T14:18:22.928238+0000","last_became_active":"2024-09-16T14:17:51.399990+0000","last_became_peered":"2024-09-16T14:17:51.399990+0000","last_unstale":"2024-09-16T14:18:22.928238+0000","last_undegraded":"2024-09-16T14:18:22.928238+0000","last_fullsized":"2024-09-16T14:18:22.928238+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:36.797120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777231+0000","last_change":"2024-09-16T14:17:55.496197+0000","last_active":"2024-09-16T14:18:22.777231+0000","last_peered":"2024-09-16T14:18:22.777231+0000","last_clean":"2024-09-16T14:18:22.777231+0000","last_became_active":"2024-09-16T14:17:55.495892+0000","last_became_peered":"2024-09-16T14:17:55.495892+0000","last_unstale":"2024-09-16T14:18:22.777231+0000","last_undegraded":"2024-09-16T14:18:22.777231+0000","last_fullsized":"2024-09-16T14:18:22.777231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:04:21.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771829+0000","last_change":"2024-09-16T14:17:57.515933+0000","last_active":"2024-09-16T14:18:22.771829+0000","last_peered":"2024-09-16T14:18:22.771829+0000","last_clean":"2024-09-16T14:18:22.771829+0000","last_became_active":"2024-09-16T14:17:57.515595+0000","last_became_peered":"2024-09-16T14:17:57.515595+0000","last_unstale":"2024-09-16T14:18:22.771829+0000","last_undegraded":"2024-09-16T14:18:22.771829+0000","last_fullsized":"2024-09-16T14:18:22.771829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:56.474292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'14","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928515+0000","last_change":"2024-09-16T14:17:53.850653+0000","last_active":"2024-09-16T14:18:22.928515+0000","last_peered":"2024-09-16T14:18:22.928515+0000","last_clean":"2024-09-16T14:18:22.928515+0000","last_became_active":"2024-09-16T14:17:53.850318+0000","last_became_peered":"2024-09-16T14:17:53.850318+0000","last_unstale":"2024-09-16T14:18:22.928515+0000","last_undegraded":"2024-09-16T14:18:22.928515+0000","last_fullsized":"2024-09-16T14:18:22.928515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:37.630005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771489+0000","last_change":"2024-09-16T14:17:55.481197+0000","last_active":"2024-09-16T14:18:22.771489+0000","last_peered":"2024-09-16T14:18:22.771489+0000","last_clean":"2024-09-16T14:18:22.771489+0000","last_became_active":"2024-09-16T14:17:51.400602+0000","last_became_peered":"2024-09-16T14:17:51.400602+0000","last_unstale":"2024-09-16T14:18:22.771489+0000","last_undegraded":"2024-09-16T14:18:22.771489+0000","last_fullsized":"2024-09-16T14:18:22.771489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:32.518153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00108147,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776767+0000","last_change":"2024-09-16T14:17:51.402806+0000","last_active":"2024-09-16T14:18:22.776767+0000","last_peered":"2024-09-16T14:18:22.776767+0000","last_clean":"2024-09-16T14:18:22.776767+0000","last_became_active":"2024-09-16T14:17:51.402531+0000","last_became_peered":"2024-09-16T14:17:51.402531+0000","last_unstale":"2024-09-16T14:18:22.776767+0000","last_undegraded":"2024-09-16T14:18:22.776767+0000","last_fullsized":"2024-09-16T14:18:22.776767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:30.290994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'188","reported_seq":338,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:18:31.645052+0000","last_change":"2024-09-16T14:17:40.182849+0000","last_active":"2024-09-16T14:18:31.645052+0000","last_peered":"2024-09-16T14:18:31.645052+0000","last_clean":"2024-09-16T14:18:31.645052+0000","last_became_active":"2024-09-16T14:17:40.182404+0000","last_became_peered":"2024-09-16T14:17:40.182404+0000","last_unstale":"2024-09-16T14:18:31.645052+0000","last_undegraded":"2024-09-16T14:18:31.645052+0000","last_fullsized":"2024-09-16T14:18:31.645052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:15:34.108483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:15:34.108483+0000","last_clean_scrub_stamp":"2024-09-16T14:15:34.108483+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:41.781469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928432+0000","last_change":"2024-09-16T14:17:55.499840+0000","last_active":"2024-09-16T14:18:22.928432+0000","last_peered":"2024-09-16T14:18:22.928432+0000","last_clean":"2024-09-16T14:18:22.928432+0000","last_became_active":"2024-09-16T14:17:55.499558+0000","last_became_peered":"2024-09-16T14:17:55.499558+0000","last_unstale":"2024-09-16T14:18:22.928432+0000","last_undegraded":"2024-09-16T14:18:22.928432+0000","last_fullsized":"2024-09-16T14:18:22.928432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:27.103751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777281+0000","last_change":"2024-09-16T14:17:57.509730+0000","last_active":"2024-09-16T14:18:22.777281+0000","last_peered":"2024-09-16T14:18:22.777281+0000","last_clean":"2024-09-16T14:18:22.777281+0000","last_became_active":"2024-09-16T14:17:57.509444+0000","last_became_peered":"2024-09-16T14:17:57.509444+0000","last_unstale":"2024-09-16T14:18:22.777281+0000","last_undegraded":"2024-09-16T14:18:22.777281+0000","last_fullsized":"2024-09-16T14:18:22.777281+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:47.956966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928186+0000","last_change":"2024-09-16T14:17:53.850605+0000","last_active":"2024-09-16T14:18:22.928186+0000","last_peered":"2024-09-16T14:18:22.928186+0000","last_clean":"2024-09-16T14:18:22.928186+0000","last_became_active":"2024-09-16T14:17:53.850276+0000","last_became_peered":"2024-09-16T14:17:53.850276+0000","last_unstale":"2024-09-16T14:18:22.928186+0000","last_undegraded":"2024-09-16T14:18:22.928186+0000","last_fullsized":"2024-09-16T14:18:22.928186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:00.311782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777167+0000","last_change":"2024-09-16T14:17:51.404540+0000","last_active":"2024-09-16T14:18:22.777167+0000","last_peered":"2024-09-16T14:18:22.777167+0000","last_clean":"2024-09-16T14:18:22.777167+0000","last_became_active":"2024-09-16T14:17:51.404124+0000","last_became_peered":"2024-09-16T14:17:51.404124+0000","last_unstale":"2024-09-16T14:18:22.777167+0000","last_undegraded":"2024-09-16T14:18:22.777167+0000","last_fullsized":"2024-09-16T14:18:22.777167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:01.026961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775758+0000","last_change":"2024-09-16T14:17:55.495331+0000","last_active":"2024-09-16T14:18:22.775758+0000","last_peered":"2024-09-16T14:18:22.775758+0000","last_clean":"2024-09-16T14:18:22.775758+0000","last_became_active":"2024-09-16T14:17:55.494932+0000","last_became_peered":"2024-09-16T14:17:55.494932+0000","last_unstale":"2024-09-16T14:18:22.775758+0000","last_undegraded":"2024-09-16T14:18:22.775758+0000","last_fullsized":"2024-09-16T14:18:22.775758+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:45.021548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928337+0000","last_change":"2024-09-16T14:17:57.516188+0000","last_active":"2024-09-16T14:18:22.928337+0000","last_peered":"2024-09-16T14:18:22.928337+0000","last_clean":"2024-09-16T14:18:22.928337+0000","last_became_active":"2024-09-16T14:17:57.515933+0000","last_became_peered":"2024-09-16T14:17:57.515933+0000","last_unstale":"2024-09-16T14:18:22.928337+0000","last_undegraded":"2024-09-16T14:18:22.928337+0000","last_fullsized":"2024-09-16T14:18:22.928337+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:15.551338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.778827+0000","last_change":"2024-09-16T14:17:53.482565+0000","last_active":"2024-09-16T14:18:22.778827+0000","last_peered":"2024-09-16T14:18:22.778827+0000","last_clean":"2024-09-16T14:18:22.778827+0000","last_became_active":"2024-09-16T14:17:53.482294+0000","last_became_peered":"2024-09-16T14:17:53.482294+0000","last_unstale":"2024-09-16T14:18:22.778827+0000","last_undegraded":"2024-09-16T14:18:22.778827+0000","last_fullsized":"2024-09-16T14:18:22.778827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:34.095826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.773041+0000","last_change":"2024-09-16T14:17:51.390862+0000","last_active":"2024-09-16T14:18:22.773041+0000","last_peered":"2024-09-16T14:18:22.773041+0000","last_clean":"2024-09-16T14:18:22.773041+0000","last_became_active":"2024-09-16T14:17:51.390494+0000","last_became_peered":"2024-09-16T14:17:51.390494+0000","last_unstale":"2024-09-16T14:18:22.773041+0000","last_undegraded":"2024-09-16T14:18:22.773041+0000","last_fullsized":"2024-09-16T14:18:22.773041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:53:33.818723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775452+0000","last_change":"2024-09-16T14:17:55.498231+0000","last_active":"2024-09-16T14:18:22.775452+0000","last_peered":"2024-09-16T14:18:22.775452+0000","last_clean":"2024-09-16T14:18:22.775452+0000","last_became_active":"2024-09-16T14:17:55.497835+0000","last_became_peered":"2024-09-16T14:17:55.497835+0000","last_unstale":"2024-09-16T14:18:22.775452+0000","last_undegraded":"2024-09-16T14:18:22.775452+0000","last_fullsized":"2024-09-16T14:18:22.775452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:25.144897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777512+0000","last_change":"2024-09-16T14:17:57.512137+0000","last_active":"2024-09-16T14:18:22.777512+0000","last_peered":"2024-09-16T14:18:22.777512+0000","last_clean":"2024-09-16T14:18:22.777512+0000","last_became_active":"2024-09-16T14:17:57.511800+0000","last_became_peered":"2024-09-16T14:17:57.511800+0000","last_unstale":"2024-09-16T14:18:22.777512+0000","last_undegraded":"2024-09-16T14:18:22.777512+0000","last_fullsized":"2024-09-16T14:18:22.777512+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:27:46.653558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776116+0000","last_change":"2024-09-16T14:17:53.850692+0000","last_active":"2024-09-16T14:18:22.776116+0000","last_peered":"2024-09-16T14:18:22.776116+0000","last_clean":"2024-09-16T14:18:22.776116+0000","last_became_active":"2024-09-16T14:17:53.850259+0000","last_became_peered":"2024-09-16T14:17:53.850259+0000","last_unstale":"2024-09-16T14:18:22.776116+0000","last_undegraded":"2024-09-16T14:18:22.776116+0000","last_fullsized":"2024-09-16T14:18:22.776116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:44.042366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.780581+0000","last_change":"2024-09-16T14:18:22.779939+0000","last_active":"2024-09-16T14:18:22.780581+0000","last_peered":"2024-09-16T14:18:22.780581+0000","last_clean":"2024-09-16T14:18:22.780581+0000","last_became_active":"2024-09-16T14:18:22.779587+0000","last_became_peered":"2024-09-16T14:18:22.779587+0000","last_unstale":"2024-09-16T14:18:22.780581+0000","last_undegraded":"2024-09-16T14:18:22.780581+0000","last_fullsized":"2024-09-16T14:18:22.780581+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:14.945046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511576+0000","last_change":"2024-09-16T14:17:55.496084+0000","last_active":"2024-09-16T14:19:04.511576+0000","last_peered":"2024-09-16T14:19:04.511576+0000","last_clean":"2024-09-16T14:19:04.511576+0000","last_became_active":"2024-09-16T14:17:55.495807+0000","last_became_peered":"2024-09-16T14:17:55.495807+0000","last_unstale":"2024-09-16T14:19:04.511576+0000","last_undegraded":"2024-09-16T14:19:04.511576+0000","last_fullsized":"2024-09-16T14:19:04.511576+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:44.423911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776349+0000","last_change":"2024-09-16T14:17:57.711145+0000","last_active":"2024-09-16T14:18:22.776349+0000","last_peered":"2024-09-16T14:18:22.776349+0000","last_clean":"2024-09-16T14:18:22.776349+0000","last_became_active":"2024-09-16T14:17:57.710861+0000","last_became_peered":"2024-09-16T14:17:57.710861+0000","last_unstale":"2024-09-16T14:18:22.776349+0000","last_undegraded":"2024-09-16T14:18:22.776349+0000","last_fullsized":"2024-09-16T14:18:22.776349+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:52.948185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771401+0000","last_change":"2024-09-16T14:17:53.477373+0000","last_active":"2024-09-16T14:18:22.771401+0000","last_peered":"2024-09-16T14:18:22.771401+0000","last_clean":"2024-09-16T14:18:22.771401+0000","last_became_active":"2024-09-16T14:17:53.477051+0000","last_became_peered":"2024-09-16T14:17:53.477051+0000","last_unstale":"2024-09-16T14:18:22.771401+0000","last_undegraded":"2024-09-16T14:18:22.771401+0000","last_fullsized":"2024-09-16T14:18:22.771401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:00.948128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772201+0000","last_change":"2024-09-16T14:17:51.395616+0000","last_active":"2024-09-16T14:18:22.772201+0000","last_peered":"2024-09-16T14:18:22.772201+0000","last_clean":"2024-09-16T14:18:22.772201+0000","last_became_active":"2024-09-16T14:17:51.395331+0000","last_became_peered":"2024-09-16T14:17:51.395331+0000","last_unstale":"2024-09-16T14:18:22.772201+0000","last_undegraded":"2024-09-16T14:18:22.772201+0000","last_fullsized":"2024-09-16T14:18:22.772201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:45.937702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775309+0000","last_change":"2024-09-16T14:17:55.491163+0000","last_active":"2024-09-16T14:18:22.775309+0000","last_peered":"2024-09-16T14:18:22.775309+0000","last_clean":"2024-09-16T14:18:22.775309+0000","last_became_active":"2024-09-16T14:17:55.490860+0000","last_became_peered":"2024-09-16T14:17:55.490860+0000","last_unstale":"2024-09-16T14:18:22.775309+0000","last_undegraded":"2024-09-16T14:18:22.775309+0000","last_fullsized":"2024-09-16T14:18:22.775309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:57.020362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771496+0000","last_change":"2024-09-16T14:17:57.514127+0000","last_active":"2024-09-16T14:18:22.771496+0000","last_peered":"2024-09-16T14:18:22.771496+0000","last_clean":"2024-09-16T14:18:22.771496+0000","last_became_active":"2024-09-16T14:17:57.513834+0000","last_became_peered":"2024-09-16T14:17:57.513834+0000","last_unstale":"2024-09-16T14:18:22.771496+0000","last_undegraded":"2024-09-16T14:18:22.771496+0000","last_fullsized":"2024-09-16T14:18:22.771496+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:34.242501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776653+0000","last_change":"2024-09-16T14:17:53.851506+0000","last_active":"2024-09-16T14:18:22.776653+0000","last_peered":"2024-09-16T14:18:22.776653+0000","last_clean":"2024-09-16T14:18:22.776653+0000","last_became_active":"2024-09-16T14:17:53.850278+0000","last_became_peered":"2024-09-16T14:17:53.850278+0000","last_unstale":"2024-09-16T14:18:22.776653+0000","last_undegraded":"2024-09-16T14:18:22.776653+0000","last_fullsized":"2024-09-16T14:18:22.776653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:30.592547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776043+0000","last_change":"2024-09-16T14:17:51.405086+0000","last_active":"2024-09-16T14:18:22.776043+0000","last_peered":"2024-09-16T14:18:22.776043+0000","last_clean":"2024-09-16T14:18:22.776043+0000","last_became_active":"2024-09-16T14:17:51.404619+0000","last_became_peered":"2024-09-16T14:17:51.404619+0000","last_unstale":"2024-09-16T14:18:22.776043+0000","last_undegraded":"2024-09-16T14:18:22.776043+0000","last_fullsized":"2024-09-16T14:18:22.776043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:17.279280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772132+0000","last_change":"2024-09-16T14:17:55.491871+0000","last_active":"2024-09-16T14:18:22.772132+0000","last_peered":"2024-09-16T14:18:22.772132+0000","last_clean":"2024-09-16T14:18:22.772132+0000","last_became_active":"2024-09-16T14:17:55.491556+0000","last_became_peered":"2024-09-16T14:17:55.491556+0000","last_unstale":"2024-09-16T14:18:22.772132+0000","last_undegraded":"2024-09-16T14:18:22.772132+0000","last_fullsized":"2024-09-16T14:18:22.772132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:04.892734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775635+0000","last_change":"2024-09-16T14:17:57.514463+0000","last_active":"2024-09-16T14:18:22.775635+0000","last_peered":"2024-09-16T14:18:22.775635+0000","last_clean":"2024-09-16T14:18:22.775635+0000","last_became_active":"2024-09-16T14:17:57.514043+0000","last_became_peered":"2024-09-16T14:17:57.514043+0000","last_unstale":"2024-09-16T14:18:22.775635+0000","last_undegraded":"2024-09-16T14:18:22.775635+0000","last_fullsized":"2024-09-16T14:18:22.775635+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:09.920931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.927926+0000","last_change":"2024-09-16T14:17:51.395406+0000","last_active":"2024-09-16T14:18:22.927926+0000","last_peered":"2024-09-16T14:18:22.927926+0000","last_clean":"2024-09-16T14:18:22.927926+0000","last_became_active":"2024-09-16T14:17:51.395077+0000","last_became_peered":"2024-09-16T14:17:51.395077+0000","last_unstale":"2024-09-16T14:18:22.927926+0000","last_undegraded":"2024-09-16T14:18:22.927926+0000","last_fullsized":"2024-09-16T14:18:22.927926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:03.552272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772280+0000","last_change":"2024-09-16T14:17:53.851047+0000","last_active":"2024-09-16T14:18:22.772280+0000","last_peered":"2024-09-16T14:18:22.772280+0000","last_clean":"2024-09-16T14:18:22.772280+0000","last_became_active":"2024-09-16T14:17:53.850750+0000","last_became_peered":"2024-09-16T14:17:53.850750+0000","last_unstale":"2024-09-16T14:18:22.772280+0000","last_undegraded":"2024-09-16T14:18:22.772280+0000","last_fullsized":"2024-09-16T14:18:22.772280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:00.046289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776999+0000","last_change":"2024-09-16T14:17:55.493178+0000","last_active":"2024-09-16T14:18:22.776999+0000","last_peered":"2024-09-16T14:18:22.776999+0000","last_clean":"2024-09-16T14:18:22.776999+0000","last_became_active":"2024-09-16T14:17:55.492894+0000","last_became_peered":"2024-09-16T14:17:55.492894+0000","last_unstale":"2024-09-16T14:18:22.776999+0000","last_undegraded":"2024-09-16T14:18:22.776999+0000","last_fullsized":"2024-09-16T14:18:22.776999+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:28:28.920343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928193+0000","last_change":"2024-09-16T14:17:57.712055+0000","last_active":"2024-09-16T14:18:22.928193+0000","last_peered":"2024-09-16T14:18:22.928193+0000","last_clean":"2024-09-16T14:18:22.928193+0000","last_became_active":"2024-09-16T14:17:57.711795+0000","last_became_peered":"2024-09-16T14:17:57.711795+0000","last_unstale":"2024-09-16T14:18:22.928193+0000","last_undegraded":"2024-09-16T14:18:22.928193+0000","last_fullsized":"2024-09-16T14:18:22.928193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:20.713477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772147+0000","last_change":"2024-09-16T14:17:53.851736+0000","last_active":"2024-09-16T14:18:22.772147+0000","last_peered":"2024-09-16T14:18:22.772147+0000","last_clean":"2024-09-16T14:18:22.772147+0000","last_became_active":"2024-09-16T14:17:53.850102+0000","last_became_peered":"2024-09-16T14:17:53.850102+0000","last_unstale":"2024-09-16T14:18:22.772147+0000","last_undegraded":"2024-09-16T14:18:22.772147+0000","last_fullsized":"2024-09-16T14:18:22.772147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:34:29.765267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776576+0000","last_change":"2024-09-16T14:17:51.397551+0000","last_active":"2024-09-16T14:18:22.776576+0000","last_peered":"2024-09-16T14:18:22.776576+0000","last_clean":"2024-09-16T14:18:22.776576+0000","last_became_active":"2024-09-16T14:17:51.397116+0000","last_became_peered":"2024-09-16T14:17:51.397116+0000","last_unstale":"2024-09-16T14:18:22.776576+0000","last_undegraded":"2024-09-16T14:18:22.776576+0000","last_fullsized":"2024-09-16T14:18:22.776576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:16.588915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511619+0000","last_change":"2024-09-16T14:17:55.494870+0000","last_active":"2024-09-16T14:19:04.511619+0000","last_peered":"2024-09-16T14:19:04.511619+0000","last_clean":"2024-09-16T14:19:04.511619+0000","last_became_active":"2024-09-16T14:17:55.494503+0000","last_became_peered":"2024-09-16T14:17:55.494503+0000","last_unstale":"2024-09-16T14:19:04.511619+0000","last_undegraded":"2024-09-16T14:19:04.511619+0000","last_fullsized":"2024-09-16T14:19:04.511619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:13.273987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776329+0000","last_change":"2024-09-16T14:17:57.512950+0000","last_active":"2024-09-16T14:18:22.776329+0000","last_peered":"2024-09-16T14:18:22.776329+0000","last_clean":"2024-09-16T14:18:22.776329+0000","last_became_active":"2024-09-16T14:17:57.512644+0000","last_became_peered":"2024-09-16T14:17:57.512644+0000","last_unstale":"2024-09-16T14:18:22.776329+0000","last_undegraded":"2024-09-16T14:18:22.776329+0000","last_fullsized":"2024-09-16T14:18:22.776329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:25:55.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776866+0000","last_change":"2024-09-16T14:17:53.850629+0000","last_active":"2024-09-16T14:18:22.776866+0000","last_peered":"2024-09-16T14:18:22.776866+0000","last_clean":"2024-09-16T14:18:22.776866+0000","last_became_active":"2024-09-16T14:17:53.850293+0000","last_became_peered":"2024-09-16T14:17:53.850293+0000","last_unstale":"2024-09-16T14:18:22.776866+0000","last_undegraded":"2024-09-16T14:18:22.776866+0000","last_fullsized":"2024-09-16T14:18:22.776866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:33.472563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.779017+0000","last_change":"2024-09-16T14:17:51.391840+0000","last_active":"2024-09-16T14:18:22.779017+0000","last_peered":"2024-09-16T14:18:22.779017+0000","last_clean":"2024-09-16T14:18:22.779017+0000","last_became_active":"2024-09-16T14:17:51.391536+0000","last_became_peered":"2024-09-16T14:17:51.391536+0000","last_unstale":"2024-09-16T14:18:22.779017+0000","last_undegraded":"2024-09-16T14:18:22.779017+0000","last_fullsized":"2024-09-16T14:18:22.779017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:49:50.059590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511444+0000","last_change":"2024-09-16T14:17:55.490014+0000","last_active":"2024-09-16T14:19:04.511444+0000","last_peered":"2024-09-16T14:19:04.511444+0000","last_clean":"2024-09-16T14:19:04.511444+0000","last_became_active":"2024-09-16T14:17:55.489674+0000","last_became_peered":"2024-09-16T14:17:55.489674+0000","last_unstale":"2024-09-16T14:19:04.511444+0000","last_undegraded":"2024-09-16T14:19:04.511444+0000","last_fullsized":"2024-09-16T14:19:04.511444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:12:49.143663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772057+0000","last_change":"2024-09-16T14:17:57.507197+0000","last_active":"2024-09-16T14:18:22.772057+0000","last_peered":"2024-09-16T14:18:22.772057+0000","last_clean":"2024-09-16T14:18:22.772057+0000","last_became_active":"2024-09-16T14:17:57.506894+0000","last_became_peered":"2024-09-16T14:17:57.506894+0000","last_unstale":"2024-09-16T14:18:22.772057+0000","last_undegraded":"2024-09-16T14:18:22.772057+0000","last_fullsized":"2024-09-16T14:18:22.772057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:29.164369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772064+0000","last_change":"2024-09-16T14:17:53.482350+0000","last_active":"2024-09-16T14:18:22.772064+0000","last_peered":"2024-09-16T14:18:22.772064+0000","last_clean":"2024-09-16T14:18:22.772064+0000","last_became_active":"2024-09-16T14:17:53.482009+0000","last_became_peered":"2024-09-16T14:17:53.482009+0000","last_unstale":"2024-09-16T14:18:22.772064+0000","last_undegraded":"2024-09-16T14:18:22.772064+0000","last_fullsized":"2024-09-16T14:18:22.772064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:28.565243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928359+0000","last_change":"2024-09-16T14:17:51.394421+0000","last_active":"2024-09-16T14:18:22.928359+0000","last_peered":"2024-09-16T14:18:22.928359+0000","last_clean":"2024-09-16T14:18:22.928359+0000","last_became_active":"2024-09-16T14:17:51.394137+0000","last_became_peered":"2024-09-16T14:17:51.394137+0000","last_unstale":"2024-09-16T14:18:22.928359+0000","last_undegraded":"2024-09-16T14:18:22.928359+0000","last_fullsized":"2024-09-16T14:18:22.928359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:54.060065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771641+0000","last_change":"2024-09-16T14:17:55.493439+0000","last_active":"2024-09-16T14:18:22.771641+0000","last_peered":"2024-09-16T14:18:22.771641+0000","last_clean":"2024-09-16T14:18:22.771641+0000","last_became_active":"2024-09-16T14:17:55.493140+0000","last_became_peered":"2024-09-16T14:17:55.493140+0000","last_unstale":"2024-09-16T14:18:22.771641+0000","last_undegraded":"2024-09-16T14:18:22.771641+0000","last_fullsized":"2024-09-16T14:18:22.771641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:28.917395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776282+0000","last_change":"2024-09-16T14:17:57.515187+0000","last_active":"2024-09-16T14:18:22.776282+0000","last_peered":"2024-09-16T14:18:22.776282+0000","last_clean":"2024-09-16T14:18:22.776282+0000","last_became_active":"2024-09-16T14:17:57.514892+0000","last_became_peered":"2024-09-16T14:17:57.514892+0000","last_unstale":"2024-09-16T14:18:22.776282+0000","last_undegraded":"2024-09-16T14:18:22.776282+0000","last_fullsized":"2024-09-16T14:18:22.776282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:28.297737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771540+0000","last_change":"2024-09-16T14:17:53.480543+0000","last_active":"2024-09-16T14:18:22.771540+0000","last_peered":"2024-09-16T14:18:22.771540+0000","last_clean":"2024-09-16T14:18:22.771540+0000","last_became_active":"2024-09-16T14:17:53.480248+0000","last_became_peered":"2024-09-16T14:17:53.480248+0000","last_unstale":"2024-09-16T14:18:22.771540+0000","last_undegraded":"2024-09-16T14:18:22.771540+0000","last_fullsized":"2024-09-16T14:18:22.771540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:40.337291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772963+0000","last_change":"2024-09-16T14:17:51.392587+0000","last_active":"2024-09-16T14:18:22.772963+0000","last_peered":"2024-09-16T14:18:22.772963+0000","last_clean":"2024-09-16T14:18:22.772963+0000","last_became_active":"2024-09-16T14:17:51.392382+0000","last_became_peered":"2024-09-16T14:17:51.392382+0000","last_unstale":"2024-09-16T14:18:22.772963+0000","last_undegraded":"2024-09-16T14:18:22.772963+0000","last_fullsized":"2024-09-16T14:18:22.772963+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:53.476072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771917+0000","last_change":"2024-09-16T14:17:55.493497+0000","last_active":"2024-09-16T14:18:22.771917+0000","last_peered":"2024-09-16T14:18:22.771917+0000","last_clean":"2024-09-16T14:18:22.771917+0000","last_became_active":"2024-09-16T14:17:55.493233+0000","last_became_peered":"2024-09-16T14:17:55.493233+0000","last_unstale":"2024-09-16T14:18:22.771917+0000","last_undegraded":"2024-09-16T14:18:22.771917+0000","last_fullsized":"2024-09-16T14:18:22.771917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:49.274458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771215+0000","last_change":"2024-09-16T14:17:57.509385+0000","last_active":"2024-09-16T14:18:22.771215+0000","last_peered":"2024-09-16T14:18:22.771215+0000","last_clean":"2024-09-16T14:18:22.771215+0000","last_became_active":"2024-09-16T14:17:57.509043+0000","last_became_peered":"2024-09-16T14:17:57.509043+0000","last_unstale":"2024-09-16T14:18:22.771215+0000","last_undegraded":"2024-09-16T14:18:22.771215+0000","last_fullsized":"2024-09-16T14:18:22.771215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:40.600301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771030+0000","last_change":"2024-09-16T14:17:53.485198+0000","last_active":"2024-09-16T14:18:22.771030+0000","last_peered":"2024-09-16T14:18:22.771030+0000","last_clean":"2024-09-16T14:18:22.771030+0000","last_became_active":"2024-09-16T14:17:53.484733+0000","last_became_peered":"2024-09-16T14:17:53.484733+0000","last_unstale":"2024-09-16T14:18:22.771030+0000","last_undegraded":"2024-09-16T14:18:22.771030+0000","last_fullsized":"2024-09-16T14:18:22.771030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:36.276380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777044+0000","last_change":"2024-09-16T14:17:51.399661+0000","last_active":"2024-09-16T14:18:22.777044+0000","last_peered":"2024-09-16T14:18:22.777044+0000","last_clean":"2024-09-16T14:18:22.777044+0000","last_became_active":"2024-09-16T14:17:51.399337+0000","last_became_peered":"2024-09-16T14:17:51.399337+0000","last_unstale":"2024-09-16T14:18:22.777044+0000","last_undegraded":"2024-09-16T14:18:22.777044+0000","last_fullsized":"2024-09-16T14:18:22.777044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:29.495774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.927728+0000","last_change":"2024-09-16T14:17:55.496036+0000","last_active":"2024-09-16T14:18:22.927728+0000","last_peered":"2024-09-16T14:18:22.927728+0000","last_clean":"2024-09-16T14:18:22.927728+0000","last_became_active":"2024-09-16T14:17:55.495687+0000","last_became_peered":"2024-09-16T14:17:55.495687+0000","last_unstale":"2024-09-16T14:18:22.927728+0000","last_undegraded":"2024-09-16T14:18:22.927728+0000","last_fullsized":"2024-09-16T14:18:22.927728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:44:17.423745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928152+0000","last_change":"2024-09-16T14:17:57.517006+0000","last_active":"2024-09-16T14:18:22.928152+0000","last_peered":"2024-09-16T14:18:22.928152+0000","last_clean":"2024-09-16T14:18:22.928152+0000","last_became_active":"2024-09-16T14:17:57.516460+0000","last_became_peered":"2024-09-16T14:17:57.516460+0000","last_unstale":"2024-09-16T14:18:22.928152+0000","last_undegraded":"2024-09-16T14:18:22.928152+0000","last_fullsized":"2024-09-16T14:18:22.928152+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:22.909755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771437+0000","last_change":"2024-09-16T14:17:53.850843+0000","last_active":"2024-09-16T14:18:22.771437+0000","last_peered":"2024-09-16T14:18:22.771437+0000","last_clean":"2024-09-16T14:18:22.771437+0000","last_became_active":"2024-09-16T14:17:53.850087+0000","last_became_peered":"2024-09-16T14:17:53.850087+0000","last_unstale":"2024-09-16T14:18:22.771437+0000","last_undegraded":"2024-09-16T14:18:22.771437+0000","last_fullsized":"2024-09-16T14:18:22.771437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:00.822052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928633+0000","last_change":"2024-09-16T14:17:51.401102+0000","last_active":"2024-09-16T14:18:22.928633+0000","last_peered":"2024-09-16T14:18:22.928633+0000","last_clean":"2024-09-16T14:18:22.928633+0000","last_became_active":"2024-09-16T14:17:51.400715+0000","last_became_peered":"2024-09-16T14:17:51.400715+0000","last_unstale":"2024-09-16T14:18:22.928633+0000","last_undegraded":"2024-09-16T14:18:22.928633+0000","last_fullsized":"2024-09-16T14:18:22.928633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:31.456651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.927845+0000","last_change":"2024-09-16T14:17:55.498006+0000","last_active":"2024-09-16T14:18:22.927845+0000","last_peered":"2024-09-16T14:18:22.927845+0000","last_clean":"2024-09-16T14:18:22.927845+0000","last_became_active":"2024-09-16T14:17:55.497676+0000","last_became_peered":"2024-09-16T14:17:55.497676+0000","last_unstale":"2024-09-16T14:18:22.927845+0000","last_undegraded":"2024-09-16T14:18:22.927845+0000","last_fullsized":"2024-09-16T14:18:22.927845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:43.597566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771419+0000","last_change":"2024-09-16T14:17:57.712962+0000","last_active":"2024-09-16T14:18:22.771419+0000","last_peered":"2024-09-16T14:18:22.771419+0000","last_clean":"2024-09-16T14:18:22.771419+0000","last_became_active":"2024-09-16T14:17:57.712636+0000","last_became_peered":"2024-09-16T14:17:57.712636+0000","last_unstale":"2024-09-16T14:18:22.771419+0000","last_undegraded":"2024-09-16T14:18:22.771419+0000","last_fullsized":"2024-09-16T14:18:22.771419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:43.967118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928359+0000","last_change":"2024-09-16T14:17:57.516711+0000","last_active":"2024-09-16T14:18:22.928359+0000","last_peered":"2024-09-16T14:18:22.928359+0000","last_clean":"2024-09-16T14:18:22.928359+0000","last_became_active":"2024-09-16T14:17:57.516232+0000","last_became_peered":"2024-09-16T14:17:57.516232+0000","last_unstale":"2024-09-16T14:18:22.928359+0000","last_undegraded":"2024-09-16T14:18:22.928359+0000","last_fullsized":"2024-09-16T14:18:22.928359+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:58.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"61'1","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776170+0000","last_change":"2024-09-16T14:17:51.396186+0000","last_active":"2024-09-16T14:18:22.776170+0000","last_peered":"2024-09-16T14:18:22.776170+0000","last_clean":"2024-09-16T14:18:22.776170+0000","last_became_active":"2024-09-16T14:17:51.395926+0000","last_became_peered":"2024-09-16T14:17:51.395926+0000","last_unstale":"2024-09-16T14:18:22.776170+0000","last_undegraded":"2024-09-16T14:18:22.776170+0000","last_fullsized":"2024-09-16T14:18:22.776170+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:02.599865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776565+0000","last_change":"2024-09-16T14:17:53.851170+0000","last_active":"2024-09-16T14:18:22.776565+0000","last_peered":"2024-09-16T14:18:22.776565+0000","last_clean":"2024-09-16T14:18:22.776565+0000","last_became_active":"2024-09-16T14:17:53.850880+0000","last_became_peered":"2024-09-16T14:17:53.850880+0000","last_unstale":"2024-09-16T14:18:22.776565+0000","last_undegraded":"2024-09-16T14:18:22.776565+0000","last_fullsized":"2024-09-16T14:18:22.776565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:13.650295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511594+0000","last_change":"2024-09-16T14:17:55.496464+0000","last_active":"2024-09-16T14:19:04.511594+0000","last_peered":"2024-09-16T14:19:04.511594+0000","last_clean":"2024-09-16T14:19:04.511594+0000","last_became_active":"2024-09-16T14:17:55.496171+0000","last_became_peered":"2024-09-16T14:17:55.496171+0000","last_unstale":"2024-09-16T14:19:04.511594+0000","last_undegraded":"2024-09-16T14:19:04.511594+0000","last_fullsized":"2024-09-16T14:19:04.511594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:06.757009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772410+0000","last_change":"2024-09-16T14:17:57.516003+0000","last_active":"2024-09-16T14:18:22.772410+0000","last_peered":"2024-09-16T14:18:22.772410+0000","last_clean":"2024-09-16T14:18:22.772410+0000","last_became_active":"2024-09-16T14:17:57.515678+0000","last_became_peered":"2024-09-16T14:17:57.515678+0000","last_unstale":"2024-09-16T14:18:22.772410+0000","last_undegraded":"2024-09-16T14:18:22.772410+0000","last_fullsized":"2024-09-16T14:18:22.772410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:04:36.333359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771661+0000","last_change":"2024-09-16T14:17:51.403154+0000","last_active":"2024-09-16T14:18:22.771661+0000","last_peered":"2024-09-16T14:18:22.771661+0000","last_clean":"2024-09-16T14:18:22.771661+0000","last_became_active":"2024-09-16T14:17:51.402356+0000","last_became_peered":"2024-09-16T14:17:51.402356+0000","last_unstale":"2024-09-16T14:18:22.771661+0000","last_undegraded":"2024-09-16T14:18:22.771661+0000","last_fullsized":"2024-09-16T14:18:22.771661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:34.757407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928602+0000","last_change":"2024-09-16T14:17:53.479115+0000","last_active":"2024-09-16T14:18:22.928602+0000","last_peered":"2024-09-16T14:18:22.928602+0000","last_clean":"2024-09-16T14:18:22.928602+0000","last_became_active":"2024-09-16T14:17:53.478772+0000","last_became_peered":"2024-09-16T14:17:53.478772+0000","last_unstale":"2024-09-16T14:18:22.928602+0000","last_undegraded":"2024-09-16T14:18:22.928602+0000","last_fullsized":"2024-09-16T14:18:22.928602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:52:44.969859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776416+0000","last_change":"2024-09-16T14:17:55.500316+0000","last_active":"2024-09-16T14:18:22.776416+0000","last_peered":"2024-09-16T14:18:22.776416+0000","last_clean":"2024-09-16T14:18:22.776416+0000","last_became_active":"2024-09-16T14:17:55.499990+0000","last_became_peered":"2024-09-16T14:17:55.499990+0000","last_unstale":"2024-09-16T14:18:22.776416+0000","last_undegraded":"2024-09-16T14:18:22.776416+0000","last_fullsized":"2024-09-16T14:18:22.776416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:19.118263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772009+0000","last_change":"2024-09-16T14:17:57.712247+0000","last_active":"2024-09-16T14:18:22.772009+0000","last_peered":"2024-09-16T14:18:22.772009+0000","last_clean":"2024-09-16T14:18:22.772009+0000","last_became_active":"2024-09-16T14:17:57.711812+0000","last_became_peered":"2024-09-16T14:17:57.711812+0000","last_unstale":"2024-09-16T14:18:22.772009+0000","last_undegraded":"2024-09-16T14:18:22.772009+0000","last_fullsized":"2024-09-16T14:18:22.772009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:40:14.572595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928254+0000","last_change":"2024-09-16T14:17:51.395020+0000","last_active":"2024-09-16T14:18:22.928254+0000","last_peered":"2024-09-16T14:18:22.928254+0000","last_clean":"2024-09-16T14:18:22.928254+0000","last_became_active":"2024-09-16T14:17:51.394737+0000","last_became_peered":"2024-09-16T14:17:51.394737+0000","last_unstale":"2024-09-16T14:18:22.928254+0000","last_undegraded":"2024-09-16T14:18:22.928254+0000","last_fullsized":"2024-09-16T14:18:22.928254+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:01.560498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":69,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.795040+0000","last_change":"2024-09-16T14:18:22.794721+0000","last_active":"2024-09-16T14:18:22.795040+0000","last_peered":"2024-09-16T14:18:22.795040+0000","last_clean":"2024-09-16T14:18:22.795040+0000","last_became_active":"2024-09-16T14:18:22.780180+0000","last_became_peered":"2024-09-16T14:18:22.780180+0000","last_unstale":"2024-09-16T14:18:22.795040+0000","last_undegraded":"2024-09-16T14:18:22.795040+0000","last_fullsized":"2024-09-16T14:18:22.795040+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:19.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772661+0000","last_change":"2024-09-16T14:17:55.497763+0000","last_active":"2024-09-16T14:18:22.772661+0000","last_peered":"2024-09-16T14:18:22.772661+0000","last_clean":"2024-09-16T14:18:22.772661+0000","last_became_active":"2024-09-16T14:17:55.497474+0000","last_became_peered":"2024-09-16T14:17:55.497474+0000","last_unstale":"2024-09-16T14:18:22.772661+0000","last_undegraded":"2024-09-16T14:18:22.772661+0000","last_fullsized":"2024-09-16T14:18:22.772661+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:06.893530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772059+0000","last_change":"2024-09-16T14:17:57.711982+0000","last_active":"2024-09-16T14:18:22.772059+0000","last_peered":"2024-09-16T14:18:22.772059+0000","last_clean":"2024-09-16T14:18:22.772059+0000","last_became_active":"2024-09-16T14:17:57.711633+0000","last_became_peered":"2024-09-16T14:17:57.711633+0000","last_unstale":"2024-09-16T14:18:22.772059+0000","last_undegraded":"2024-09-16T14:18:22.772059+0000","last_fullsized":"2024-09-16T14:18:22.772059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:44.334874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771818+0000","last_change":"2024-09-16T14:17:51.398026+0000","last_active":"2024-09-16T14:18:22.771818+0000","last_peered":"2024-09-16T14:18:22.771818+0000","last_clean":"2024-09-16T14:18:22.771818+0000","last_became_active":"2024-09-16T14:17:51.397755+0000","last_became_peered":"2024-09-16T14:17:51.397755+0000","last_unstale":"2024-09-16T14:18:22.771818+0000","last_undegraded":"2024-09-16T14:18:22.771818+0000","last_fullsized":"2024-09-16T14:18:22.771818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:02.992907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.787594+0000","last_change":"2024-09-16T14:18:22.787153+0000","last_active":"2024-09-16T14:18:22.787594+0000","last_peered":"2024-09-16T14:18:22.787594+0000","last_clean":"2024-09-16T14:18:22.787594+0000","last_became_active":"2024-09-16T14:18:22.778551+0000","last_became_peered":"2024-09-16T14:18:22.778551+0000","last_unstale":"2024-09-16T14:18:22.787594+0000","last_undegraded":"2024-09-16T14:18:22.787594+0000","last_fullsized":"2024-09-16T14:18:22.787594+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:12.189019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771614+0000","last_change":"2024-09-16T14:17:55.493044+0000","last_active":"2024-09-16T14:18:22.771614+0000","last_peered":"2024-09-16T14:18:22.771614+0000","last_clean":"2024-09-16T14:18:22.771614+0000","last_became_active":"2024-09-16T14:17:55.492764+0000","last_became_peered":"2024-09-16T14:17:55.492764+0000","last_unstale":"2024-09-16T14:18:22.771614+0000","last_undegraded":"2024-09-16T14:18:22.771614+0000","last_fullsized":"2024-09-16T14:18:22.771614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:15.614810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":67,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201302,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32484,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708572,"statfs":{"total":95990841344,"available":95957577728,"internally_reserved":0,"allocated":1404928,"data_stored":928340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60199999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.54000000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495642,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32460,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708596,"statfs":{"total":95990841344,"available":95957602304,"internally_reserved":0,"allocated":1380352,"data_stored":927093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67300000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.45900000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42399999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51100000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31892,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709164,"statfs":{"total":95990841344,"available":95958183936,"internally_reserved":0,"allocated":794624,"data_stored":337401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.622}]},{"osd":1,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67600000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.628},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.27200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.40899999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.44600000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.46600000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709148,"statfs":{"total":95990841344,"available":95958167552,"internally_reserved":0,"allocated":811008,"data_stored":337810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.35099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66000000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.499},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.89700000000000002,"15min":0.96899999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":8.2189999999999994,"15min":8.2189999999999994},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.89100000000000001,"15min":0.95599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66800000000000004,"5min":8.234,"15min":8.234},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55800000000000005,"15min":0.56699999999999995},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66000000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54700000000000004,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70399999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.89900000000000002,"15min":0.96799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.61799999999999999,"5min":8.2680000000000007,"15min":8.2680000000000007},"last":0.437}]},{"osd":5,"last update":"Mon Sep 16 14:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.34799999999999998},"min":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.46500000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44700000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.48499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31928,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709128,"statfs":{"total":95990841344,"available":95958147072,"internally_reserved":0,"allocated":831488,"data_stored":338628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.50800000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.435},{"interface":"front","average":{"1min":0.371,"5min":0.40999999999999998,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29599999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39200000000000002,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48199999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.433,"15min":0.44900000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.379},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.437,"15min":0.44400000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52200000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40100000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.59599999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713256,"statfs":{"total":95990841344,"available":95962374144,"internally_reserved":0,"allocated":798720,"data_stored":337531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57799999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41399999999999998,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.31}]},{"osd":1,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40699999999999997,"15min":0.40400000000000003},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:19:06 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.435,"15min":0.435},"max":{"1min":0.89600000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.623,"15min":0.626},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79900000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77900000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31912,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709144,"statfs":{"total":95990841344,"available":95958163456,"internally_reserved":0,"allocated":815104,"data_stored":337891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34300000000000003,"15min":0.33000000000000002},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.48799999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.37,"15min":0.34799999999999998},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.313}]},{"osd":2,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.27700000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.41799999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.46000000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55500000000000005}]},{"osd":5,"last update":"Mon Sep 16 14:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Sep 16 14:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32480,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708576,"statfs":{"total":95990841344,"available":95957581824,"internally_reserved":0,"allocated":1396736,"data_stored":928205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.34200000000000003,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.495}]},{"osd":2,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.26400000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54000000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.372},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.36399999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.62,"15min":0.622},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.79000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:19:13.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T14:19:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:13.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-mon[38187]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:13.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-mon[38187]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:19:13.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-mon[25528]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:13.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-mon[25528]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 podman[78548]: 2024-09-16 14:19:13.262809031 +0000 UTC m=+1.789611036 container init c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 podman[78548]: 2024-09-16 14:19:13.266604746 +0000 UTC m=+1.793406759 container start c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.283Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.283Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.284Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.12 port=9094 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.285Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.311Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.311Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.314Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:13.314Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 bash[78548]: c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c 2024-09-16T14:19:13.528 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:13 smithi012 systemd[1]: Started Ceph alertmanager.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:13.837 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:13 smithi189 ceph-mon[30490]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:13.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:13 smithi189 ceph-mon[30490]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:19:13.867 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:13 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:13] ENGINE Bus STOPPING 2024-09-16T14:19:14.166 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T14:19:14.166 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:14] ENGINE Bus STOPPED 2024-09-16T14:19:14.166 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:14] ENGINE Bus STARTING 2024-09-16T14:19:14.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:19:13 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:19:14.432 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:14] ENGINE Serving on http://:::9283 2024-09-16T14:19:14.432 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:14] ENGINE Bus STARTED 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.715 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T14:19:14.717 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:14 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:14.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T14:19:14.839 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:14 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:15.092 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:19:15.094 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T14:19:15.679 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[38187]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T14:19:15.679 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[38187]: Deploying daemon grafana.a on smithi189 2024-09-16T14:19:15.679 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[38187]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:15.679 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:15.679 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:15.679 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:15.680 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:15.286Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000375396s 2024-09-16T14:19:15.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[25528]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T14:19:15.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[25528]: Deploying daemon grafana.a on smithi189 2024-09-16T14:19:15.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[25528]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:15.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:15.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:15.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:15 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:15.701 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T14:19:13.587148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1023,"num_read_kb":866,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1771764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":252864,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":749675584,"statfs":{"total":767926730752,"available":767667798016,"internally_reserved":0,"allocated":8232960,"data_stored":4472899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":250596921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771329+0000","last_change":"2024-09-16T14:17:57.712867+0000","last_active":"2024-09-16T14:18:22.771329+0000","last_peered":"2024-09-16T14:18:22.771329+0000","last_clean":"2024-09-16T14:18:22.771329+0000","last_became_active":"2024-09-16T14:17:57.712556+0000","last_became_peered":"2024-09-16T14:17:57.712556+0000","last_unstale":"2024-09-16T14:18:22.771329+0000","last_undegraded":"2024-09-16T14:18:22.771329+0000","last_fullsized":"2024-09-16T14:18:22.771329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:58.734845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.778540+0000","last_change":"2024-09-16T14:17:51.397770+0000","last_active":"2024-09-16T14:18:22.778540+0000","last_peered":"2024-09-16T14:18:22.778540+0000","last_clean":"2024-09-16T14:18:22.778540+0000","last_became_active":"2024-09-16T14:17:51.397492+0000","last_became_peered":"2024-09-16T14:17:51.397492+0000","last_unstale":"2024-09-16T14:18:22.778540+0000","last_undegraded":"2024-09-16T14:18:22.778540+0000","last_fullsized":"2024-09-16T14:18:22.778540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:41.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771541+0000","last_change":"2024-09-16T14:17:53.473856+0000","last_active":"2024-09-16T14:18:22.771541+0000","last_peered":"2024-09-16T14:18:22.771541+0000","last_clean":"2024-09-16T14:18:22.771541+0000","last_became_active":"2024-09-16T14:17:53.473534+0000","last_became_peered":"2024-09-16T14:17:53.473534+0000","last_unstale":"2024-09-16T14:18:22.771541+0000","last_undegraded":"2024-09-16T14:18:22.771541+0000","last_fullsized":"2024-09-16T14:18:22.771541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:38:18.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771346+0000","last_change":"2024-09-16T14:17:55.493162+0000","last_active":"2024-09-16T14:18:22.771346+0000","last_peered":"2024-09-16T14:18:22.771346+0000","last_clean":"2024-09-16T14:18:22.771346+0000","last_became_active":"2024-09-16T14:17:55.492888+0000","last_became_peered":"2024-09-16T14:17:55.492888+0000","last_unstale":"2024-09-16T14:18:22.771346+0000","last_undegraded":"2024-09-16T14:18:22.771346+0000","last_fullsized":"2024-09-16T14:18:22.771346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:28.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772874+0000","last_change":"2024-09-16T14:17:51.403637+0000","last_active":"2024-09-16T14:18:22.772874+0000","last_peered":"2024-09-16T14:18:22.772874+0000","last_clean":"2024-09-16T14:18:22.772874+0000","last_became_active":"2024-09-16T14:17:51.403353+0000","last_became_peered":"2024-09-16T14:17:51.403353+0000","last_unstale":"2024-09-16T14:18:22.772874+0000","last_undegraded":"2024-09-16T14:18:22.772874+0000","last_fullsized":"2024-09-16T14:18:22.772874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:59.668377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771354+0000","last_change":"2024-09-16T14:17:53.484168+0000","last_active":"2024-09-16T14:18:22.771354+0000","last_peered":"2024-09-16T14:18:22.771354+0000","last_clean":"2024-09-16T14:18:22.771354+0000","last_became_active":"2024-09-16T14:17:53.483661+0000","last_became_peered":"2024-09-16T14:17:53.483661+0000","last_unstale":"2024-09-16T14:18:22.771354+0000","last_undegraded":"2024-09-16T14:18:22.771354+0000","last_fullsized":"2024-09-16T14:18:22.771354+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:46.765389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771625+0000","last_change":"2024-09-16T14:17:55.492869+0000","last_active":"2024-09-16T14:18:22.771625+0000","last_peered":"2024-09-16T14:18:22.771625+0000","last_clean":"2024-09-16T14:18:22.771625+0000","last_became_active":"2024-09-16T14:17:55.492586+0000","last_became_peered":"2024-09-16T14:17:55.492586+0000","last_unstale":"2024-09-16T14:18:22.771625+0000","last_undegraded":"2024-09-16T14:18:22.771625+0000","last_fullsized":"2024-09-16T14:18:22.771625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:45.289151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771646+0000","last_change":"2024-09-16T14:17:57.512096+0000","last_active":"2024-09-16T14:18:22.771646+0000","last_peered":"2024-09-16T14:18:22.771646+0000","last_clean":"2024-09-16T14:18:22.771646+0000","last_became_active":"2024-09-16T14:17:57.511727+0000","last_became_peered":"2024-09-16T14:17:57.511727+0000","last_unstale":"2024-09-16T14:18:22.771646+0000","last_undegraded":"2024-09-16T14:18:22.771646+0000","last_fullsized":"2024-09-16T14:18:22.771646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:30.564310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928687+0000","last_change":"2024-09-16T14:17:51.403697+0000","last_active":"2024-09-16T14:18:22.928687+0000","last_peered":"2024-09-16T14:18:22.928687+0000","last_clean":"2024-09-16T14:18:22.928687+0000","last_became_active":"2024-09-16T14:17:51.403443+0000","last_became_peered":"2024-09-16T14:17:51.403443+0000","last_unstale":"2024-09-16T14:18:22.928687+0000","last_undegraded":"2024-09-16T14:18:22.928687+0000","last_fullsized":"2024-09-16T14:18:22.928687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:26.861141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771541+0000","last_change":"2024-09-16T14:17:53.484042+0000","last_active":"2024-09-16T14:18:22.771541+0000","last_peered":"2024-09-16T14:18:22.771541+0000","last_clean":"2024-09-16T14:18:22.771541+0000","last_became_active":"2024-09-16T14:17:53.483630+0000","last_became_peered":"2024-09-16T14:17:53.483630+0000","last_unstale":"2024-09-16T14:18:22.771541+0000","last_undegraded":"2024-09-16T14:18:22.771541+0000","last_fullsized":"2024-09-16T14:18:22.771541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:21.900570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928225+0000","last_change":"2024-09-16T14:17:55.497028+0000","last_active":"2024-09-16T14:18:22.928225+0000","last_peered":"2024-09-16T14:18:22.928225+0000","last_clean":"2024-09-16T14:18:22.928225+0000","last_became_active":"2024-09-16T14:17:55.496751+0000","last_became_peered":"2024-09-16T14:17:55.496751+0000","last_unstale":"2024-09-16T14:18:22.928225+0000","last_undegraded":"2024-09-16T14:18:22.928225+0000","last_fullsized":"2024-09-16T14:18:22.928225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:59.979638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776349+0000","last_change":"2024-09-16T14:17:57.510856+0000","last_active":"2024-09-16T14:18:22.776349+0000","last_peered":"2024-09-16T14:18:22.776349+0000","last_clean":"2024-09-16T14:18:22.776349+0000","last_became_active":"2024-09-16T14:17:57.510529+0000","last_became_peered":"2024-09-16T14:17:57.510529+0000","last_unstale":"2024-09-16T14:18:22.776349+0000","last_undegraded":"2024-09-16T14:18:22.776349+0000","last_fullsized":"2024-09-16T14:18:22.776349+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:39.547665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928574+0000","last_change":"2024-09-16T14:17:51.400901+0000","last_active":"2024-09-16T14:18:22.928574+0000","last_peered":"2024-09-16T14:18:22.928574+0000","last_clean":"2024-09-16T14:18:22.928574+0000","last_became_active":"2024-09-16T14:17:51.400635+0000","last_became_peered":"2024-09-16T14:17:51.400635+0000","last_unstale":"2024-09-16T14:18:22.928574+0000","last_undegraded":"2024-09-16T14:18:22.928574+0000","last_fullsized":"2024-09-16T14:18:22.928574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:34.534106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771908+0000","last_change":"2024-09-16T14:17:53.471486+0000","last_active":"2024-09-16T14:18:22.771908+0000","last_peered":"2024-09-16T14:18:22.771908+0000","last_clean":"2024-09-16T14:18:22.771908+0000","last_became_active":"2024-09-16T14:17:53.470940+0000","last_became_peered":"2024-09-16T14:17:53.470940+0000","last_unstale":"2024-09-16T14:18:22.771908+0000","last_undegraded":"2024-09-16T14:18:22.771908+0000","last_fullsized":"2024-09-16T14:18:22.771908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:39.782219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776605+0000","last_change":"2024-09-16T14:17:55.494933+0000","last_active":"2024-09-16T14:18:22.776605+0000","last_peered":"2024-09-16T14:18:22.776605+0000","last_clean":"2024-09-16T14:18:22.776605+0000","last_became_active":"2024-09-16T14:17:55.494614+0000","last_became_peered":"2024-09-16T14:17:55.494614+0000","last_unstale":"2024-09-16T14:18:22.776605+0000","last_undegraded":"2024-09-16T14:18:22.776605+0000","last_fullsized":"2024-09-16T14:18:22.776605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:14.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776538+0000","last_change":"2024-09-16T14:17:57.519096+0000","last_active":"2024-09-16T14:18:22.776538+0000","last_peered":"2024-09-16T14:18:22.776538+0000","last_clean":"2024-09-16T14:18:22.776538+0000","last_became_active":"2024-09-16T14:17:57.518424+0000","last_became_peered":"2024-09-16T14:17:57.518424+0000","last_unstale":"2024-09-16T14:18:22.776538+0000","last_undegraded":"2024-09-16T14:18:22.776538+0000","last_fullsized":"2024-09-16T14:18:22.776538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:26.662229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771913+0000","last_change":"2024-09-16T14:17:53.480478+0000","last_active":"2024-09-16T14:18:22.771913+0000","last_peered":"2024-09-16T14:18:22.771913+0000","last_clean":"2024-09-16T14:18:22.771913+0000","last_became_active":"2024-09-16T14:17:53.480159+0000","last_became_peered":"2024-09-16T14:17:53.480159+0000","last_unstale":"2024-09-16T14:18:22.771913+0000","last_undegraded":"2024-09-16T14:18:22.771913+0000","last_fullsized":"2024-09-16T14:18:22.771913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:50:54.318096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928713+0000","last_change":"2024-09-16T14:17:51.394867+0000","last_active":"2024-09-16T14:18:22.928713+0000","last_peered":"2024-09-16T14:18:22.928713+0000","last_clean":"2024-09-16T14:18:22.928713+0000","last_became_active":"2024-09-16T14:17:51.394341+0000","last_became_peered":"2024-09-16T14:17:51.394341+0000","last_unstale":"2024-09-16T14:18:22.928713+0000","last_undegraded":"2024-09-16T14:18:22.928713+0000","last_fullsized":"2024-09-16T14:18:22.928713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:41.051615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771919+0000","last_change":"2024-09-16T14:17:55.498471+0000","last_active":"2024-09-16T14:18:22.771919+0000","last_peered":"2024-09-16T14:18:22.771919+0000","last_clean":"2024-09-16T14:18:22.771919+0000","last_became_active":"2024-09-16T14:17:55.496725+0000","last_became_peered":"2024-09-16T14:17:55.496725+0000","last_unstale":"2024-09-16T14:18:22.771919+0000","last_undegraded":"2024-09-16T14:18:22.771919+0000","last_fullsized":"2024-09-16T14:18:22.771919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:28.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777116+0000","last_change":"2024-09-16T14:17:57.512976+0000","last_active":"2024-09-16T14:18:22.777116+0000","last_peered":"2024-09-16T14:18:22.777116+0000","last_clean":"2024-09-16T14:18:22.777116+0000","last_became_active":"2024-09-16T14:17:57.512585+0000","last_became_peered":"2024-09-16T14:17:57.512585+0000","last_unstale":"2024-09-16T14:18:22.777116+0000","last_undegraded":"2024-09-16T14:18:22.777116+0000","last_fullsized":"2024-09-16T14:18:22.777116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:09:38.889476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771678+0000","last_change":"2024-09-16T14:17:53.470204+0000","last_active":"2024-09-16T14:18:22.771678+0000","last_peered":"2024-09-16T14:18:22.771678+0000","last_clean":"2024-09-16T14:18:22.771678+0000","last_became_active":"2024-09-16T14:17:53.469881+0000","last_became_peered":"2024-09-16T14:17:53.469881+0000","last_unstale":"2024-09-16T14:18:22.771678+0000","last_undegraded":"2024-09-16T14:18:22.771678+0000","last_fullsized":"2024-09-16T14:18:22.771678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:45.758115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772303+0000","last_change":"2024-09-16T14:17:51.399418+0000","last_active":"2024-09-16T14:18:22.772303+0000","last_peered":"2024-09-16T14:18:22.772303+0000","last_clean":"2024-09-16T14:18:22.772303+0000","last_became_active":"2024-09-16T14:17:51.399139+0000","last_became_peered":"2024-09-16T14:17:51.399139+0000","last_unstale":"2024-09-16T14:18:22.772303+0000","last_undegraded":"2024-09-16T14:18:22.772303+0000","last_fullsized":"2024-09-16T14:18:22.772303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:29.393614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511431+0000","last_change":"2024-09-16T14:17:55.498938+0000","last_active":"2024-09-16T14:19:04.511431+0000","last_peered":"2024-09-16T14:19:04.511431+0000","last_clean":"2024-09-16T14:19:04.511431+0000","last_became_active":"2024-09-16T14:17:55.498521+0000","last_became_peered":"2024-09-16T14:17:55.498521+0000","last_unstale":"2024-09-16T14:19:04.511431+0000","last_undegraded":"2024-09-16T14:19:04.511431+0000","last_fullsized":"2024-09-16T14:19:04.511431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:21.593867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771930+0000","last_change":"2024-09-16T14:17:57.516399+0000","last_active":"2024-09-16T14:18:22.771930+0000","last_peered":"2024-09-16T14:18:22.771930+0000","last_clean":"2024-09-16T14:18:22.771930+0000","last_became_active":"2024-09-16T14:17:57.516111+0000","last_became_peered":"2024-09-16T14:17:57.516111+0000","last_unstale":"2024-09-16T14:18:22.771930+0000","last_undegraded":"2024-09-16T14:18:22.771930+0000","last_fullsized":"2024-09-16T14:18:22.771930+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:53.039153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772678+0000","last_change":"2024-09-16T14:17:53.481417+0000","last_active":"2024-09-16T14:18:22.772678+0000","last_peered":"2024-09-16T14:18:22.772678+0000","last_clean":"2024-09-16T14:18:22.772678+0000","last_became_active":"2024-09-16T14:17:53.479685+0000","last_became_peered":"2024-09-16T14:17:53.479685+0000","last_unstale":"2024-09-16T14:18:22.772678+0000","last_undegraded":"2024-09-16T14:18:22.772678+0000","last_fullsized":"2024-09-16T14:18:22.772678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:10.649254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771771+0000","last_change":"2024-09-16T14:17:51.395131+0000","last_active":"2024-09-16T14:18:22.771771+0000","last_peered":"2024-09-16T14:18:22.771771+0000","last_clean":"2024-09-16T14:18:22.771771+0000","last_became_active":"2024-09-16T14:17:51.394857+0000","last_became_peered":"2024-09-16T14:17:51.394857+0000","last_unstale":"2024-09-16T14:18:22.771771+0000","last_undegraded":"2024-09-16T14:18:22.771771+0000","last_fullsized":"2024-09-16T14:18:22.771771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:29.471562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511587+0000","last_change":"2024-09-16T14:17:55.490831+0000","last_active":"2024-09-16T14:19:04.511587+0000","last_peered":"2024-09-16T14:19:04.511587+0000","last_clean":"2024-09-16T14:19:04.511587+0000","last_became_active":"2024-09-16T14:17:55.490362+0000","last_became_peered":"2024-09-16T14:17:55.490362+0000","last_unstale":"2024-09-16T14:19:04.511587+0000","last_undegraded":"2024-09-16T14:19:04.511587+0000","last_fullsized":"2024-09-16T14:19:04.511587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:31.796055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776728+0000","last_change":"2024-09-16T14:17:57.515312+0000","last_active":"2024-09-16T14:18:22.776728+0000","last_peered":"2024-09-16T14:18:22.776728+0000","last_clean":"2024-09-16T14:18:22.776728+0000","last_became_active":"2024-09-16T14:17:57.515044+0000","last_became_peered":"2024-09-16T14:17:57.515044+0000","last_unstale":"2024-09-16T14:18:22.776728+0000","last_undegraded":"2024-09-16T14:18:22.776728+0000","last_fullsized":"2024-09-16T14:18:22.776728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:33.969061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772106+0000","last_change":"2024-09-16T14:17:53.477474+0000","last_active":"2024-09-16T14:18:22.772106+0000","last_peered":"2024-09-16T14:18:22.772106+0000","last_clean":"2024-09-16T14:18:22.772106+0000","last_became_active":"2024-09-16T14:17:53.477144+0000","last_became_peered":"2024-09-16T14:17:53.477144+0000","last_unstale":"2024-09-16T14:18:22.772106+0000","last_undegraded":"2024-09-16T14:18:22.772106+0000","last_fullsized":"2024-09-16T14:18:22.772106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:59.759550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928161+0000","last_change":"2024-09-16T14:17:51.399942+0000","last_active":"2024-09-16T14:18:22.928161+0000","last_peered":"2024-09-16T14:18:22.928161+0000","last_clean":"2024-09-16T14:18:22.928161+0000","last_became_active":"2024-09-16T14:17:51.399586+0000","last_became_peered":"2024-09-16T14:17:51.399586+0000","last_unstale":"2024-09-16T14:18:22.928161+0000","last_undegraded":"2024-09-16T14:18:22.928161+0000","last_fullsized":"2024-09-16T14:18:22.928161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:16.495921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777084+0000","last_change":"2024-09-16T14:17:55.496534+0000","last_active":"2024-09-16T14:18:22.777084+0000","last_peered":"2024-09-16T14:18:22.777084+0000","last_clean":"2024-09-16T14:18:22.777084+0000","last_became_active":"2024-09-16T14:17:55.496189+0000","last_became_peered":"2024-09-16T14:17:55.496189+0000","last_unstale":"2024-09-16T14:18:22.777084+0000","last_undegraded":"2024-09-16T14:18:22.777084+0000","last_fullsized":"2024-09-16T14:18:22.777084+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:52.395816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771669+0000","last_change":"2024-09-16T14:17:57.712103+0000","last_active":"2024-09-16T14:18:22.771669+0000","last_peered":"2024-09-16T14:18:22.771669+0000","last_clean":"2024-09-16T14:18:22.771669+0000","last_became_active":"2024-09-16T14:17:57.711600+0000","last_became_peered":"2024-09-16T14:17:57.711600+0000","last_unstale":"2024-09-16T14:18:22.771669+0000","last_undegraded":"2024-09-16T14:18:22.771669+0000","last_fullsized":"2024-09-16T14:18:22.771669+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:37:34.394514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771646+0000","last_change":"2024-09-16T14:17:53.471476+0000","last_active":"2024-09-16T14:18:22.771646+0000","last_peered":"2024-09-16T14:18:22.771646+0000","last_clean":"2024-09-16T14:18:22.771646+0000","last_became_active":"2024-09-16T14:17:53.471147+0000","last_became_peered":"2024-09-16T14:17:53.471147+0000","last_unstale":"2024-09-16T14:18:22.771646+0000","last_undegraded":"2024-09-16T14:18:22.771646+0000","last_fullsized":"2024-09-16T14:18:22.771646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:29.051796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928673+0000","last_change":"2024-09-16T14:17:51.400293+0000","last_active":"2024-09-16T14:18:22.928673+0000","last_peered":"2024-09-16T14:18:22.928673+0000","last_clean":"2024-09-16T14:18:22.928673+0000","last_became_active":"2024-09-16T14:17:51.400019+0000","last_became_peered":"2024-09-16T14:17:51.400019+0000","last_unstale":"2024-09-16T14:18:22.928673+0000","last_undegraded":"2024-09-16T14:18:22.928673+0000","last_fullsized":"2024-09-16T14:18:22.928673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:37.720347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511428+0000","last_change":"2024-09-16T14:17:55.489742+0000","last_active":"2024-09-16T14:19:04.511428+0000","last_peered":"2024-09-16T14:19:04.511428+0000","last_clean":"2024-09-16T14:19:04.511428+0000","last_became_active":"2024-09-16T14:17:55.489409+0000","last_became_peered":"2024-09-16T14:17:55.489409+0000","last_unstale":"2024-09-16T14:19:04.511428+0000","last_undegraded":"2024-09-16T14:19:04.511428+0000","last_fullsized":"2024-09-16T14:19:04.511428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:14.493465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928431+0000","last_change":"2024-09-16T14:17:57.711640+0000","last_active":"2024-09-16T14:18:22.928431+0000","last_peered":"2024-09-16T14:18:22.928431+0000","last_clean":"2024-09-16T14:18:22.928431+0000","last_became_active":"2024-09-16T14:17:57.711354+0000","last_became_peered":"2024-09-16T14:17:57.711354+0000","last_unstale":"2024-09-16T14:18:22.928431+0000","last_undegraded":"2024-09-16T14:18:22.928431+0000","last_fullsized":"2024-09-16T14:18:22.928431+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:11:03.914929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771141+0000","last_change":"2024-09-16T14:17:53.850576+0000","last_active":"2024-09-16T14:18:22.771141+0000","last_peered":"2024-09-16T14:18:22.771141+0000","last_clean":"2024-09-16T14:18:22.771141+0000","last_became_active":"2024-09-16T14:17:53.850315+0000","last_became_peered":"2024-09-16T14:17:53.850315+0000","last_unstale":"2024-09-16T14:18:22.771141+0000","last_undegraded":"2024-09-16T14:18:22.771141+0000","last_fullsized":"2024-09-16T14:18:22.771141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:51.795184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.773078+0000","last_change":"2024-09-16T14:17:51.393041+0000","last_active":"2024-09-16T14:18:22.773078+0000","last_peered":"2024-09-16T14:18:22.773078+0000","last_clean":"2024-09-16T14:18:22.773078+0000","last_became_active":"2024-09-16T14:17:51.392504+0000","last_became_peered":"2024-09-16T14:17:51.392504+0000","last_unstale":"2024-09-16T14:18:22.773078+0000","last_undegraded":"2024-09-16T14:18:22.773078+0000","last_fullsized":"2024-09-16T14:18:22.773078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:24.307139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771409+0000","last_change":"2024-09-16T14:17:55.492408+0000","last_active":"2024-09-16T14:18:22.771409+0000","last_peered":"2024-09-16T14:18:22.771409+0000","last_clean":"2024-09-16T14:18:22.771409+0000","last_became_active":"2024-09-16T14:17:55.492127+0000","last_became_peered":"2024-09-16T14:17:55.492127+0000","last_unstale":"2024-09-16T14:18:22.771409+0000","last_undegraded":"2024-09-16T14:18:22.771409+0000","last_fullsized":"2024-09-16T14:18:22.771409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:29.480789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771826+0000","last_change":"2024-09-16T14:17:57.516429+0000","last_active":"2024-09-16T14:18:22.771826+0000","last_peered":"2024-09-16T14:18:22.771826+0000","last_clean":"2024-09-16T14:18:22.771826+0000","last_became_active":"2024-09-16T14:17:57.516125+0000","last_became_peered":"2024-09-16T14:17:57.516125+0000","last_unstale":"2024-09-16T14:18:22.771826+0000","last_undegraded":"2024-09-16T14:18:22.771826+0000","last_fullsized":"2024-09-16T14:18:22.771826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:01:59.715466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":55,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.778943+0000","last_change":"2024-09-16T14:17:53.850520+0000","last_active":"2024-09-16T14:18:22.778943+0000","last_peered":"2024-09-16T14:18:22.778943+0000","last_clean":"2024-09-16T14:18:22.778943+0000","last_became_active":"2024-09-16T14:17:53.850092+0000","last_became_peered":"2024-09-16T14:17:53.850092+0000","last_unstale":"2024-09-16T14:18:22.778943+0000","last_undegraded":"2024-09-16T14:18:22.778943+0000","last_fullsized":"2024-09-16T14:18:22.778943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:04.790941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928768+0000","last_change":"2024-09-16T14:17:51.403722+0000","last_active":"2024-09-16T14:18:22.928768+0000","last_peered":"2024-09-16T14:18:22.928768+0000","last_clean":"2024-09-16T14:18:22.928768+0000","last_became_active":"2024-09-16T14:17:51.403464+0000","last_became_peered":"2024-09-16T14:17:51.403464+0000","last_unstale":"2024-09-16T14:18:22.928768+0000","last_undegraded":"2024-09-16T14:18:22.928768+0000","last_fullsized":"2024-09-16T14:18:22.928768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:09.494838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928087+0000","last_change":"2024-09-16T14:17:55.495768+0000","last_active":"2024-09-16T14:18:22.928087+0000","last_peered":"2024-09-16T14:18:22.928087+0000","last_clean":"2024-09-16T14:18:22.928087+0000","last_became_active":"2024-09-16T14:17:55.495463+0000","last_became_peered":"2024-09-16T14:17:55.495463+0000","last_unstale":"2024-09-16T14:18:22.928087+0000","last_undegraded":"2024-09-16T14:18:22.928087+0000","last_fullsized":"2024-09-16T14:18:22.928087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:17.943562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771079+0000","last_change":"2024-09-16T14:17:57.711945+0000","last_active":"2024-09-16T14:18:22.771079+0000","last_peered":"2024-09-16T14:18:22.771079+0000","last_clean":"2024-09-16T14:18:22.771079+0000","last_became_active":"2024-09-16T14:17:57.711588+0000","last_became_peered":"2024-09-16T14:17:57.711588+0000","last_unstale":"2024-09-16T14:18:22.771079+0000","last_undegraded":"2024-09-16T14:18:22.771079+0000","last_fullsized":"2024-09-16T14:18:22.771079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:52.132678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772157+0000","last_change":"2024-09-16T14:17:53.481354+0000","last_active":"2024-09-16T14:18:22.772157+0000","last_peered":"2024-09-16T14:18:22.772157+0000","last_clean":"2024-09-16T14:18:22.772157+0000","last_became_active":"2024-09-16T14:17:53.480091+0000","last_became_peered":"2024-09-16T14:17:53.480091+0000","last_unstale":"2024-09-16T14:18:22.772157+0000","last_undegraded":"2024-09-16T14:18:22.772157+0000","last_fullsized":"2024-09-16T14:18:22.772157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:44.963701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.773322+0000","last_change":"2024-09-16T14:17:51.396471+0000","last_active":"2024-09-16T14:18:22.773322+0000","last_peered":"2024-09-16T14:18:22.773322+0000","last_clean":"2024-09-16T14:18:22.773322+0000","last_became_active":"2024-09-16T14:17:51.396185+0000","last_became_peered":"2024-09-16T14:17:51.396185+0000","last_unstale":"2024-09-16T14:18:22.773322+0000","last_undegraded":"2024-09-16T14:18:22.773322+0000","last_fullsized":"2024-09-16T14:18:22.773322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:55.971443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511408+0000","last_change":"2024-09-16T14:17:55.502290+0000","last_active":"2024-09-16T14:19:04.511408+0000","last_peered":"2024-09-16T14:19:04.511408+0000","last_clean":"2024-09-16T14:19:04.511408+0000","last_became_active":"2024-09-16T14:17:55.501860+0000","last_became_peered":"2024-09-16T14:17:55.501860+0000","last_unstale":"2024-09-16T14:19:04.511408+0000","last_undegraded":"2024-09-16T14:19:04.511408+0000","last_fullsized":"2024-09-16T14:19:04.511408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:31.686145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776386+0000","last_change":"2024-09-16T14:17:57.518096+0000","last_active":"2024-09-16T14:18:22.776386+0000","last_peered":"2024-09-16T14:18:22.776386+0000","last_clean":"2024-09-16T14:18:22.776386+0000","last_became_active":"2024-09-16T14:17:57.517013+0000","last_became_peered":"2024-09-16T14:17:57.517013+0000","last_unstale":"2024-09-16T14:18:22.776386+0000","last_undegraded":"2024-09-16T14:18:22.776386+0000","last_fullsized":"2024-09-16T14:18:22.776386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:08.512697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":83,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511421+0000","last_change":"2024-09-16T14:17:53.481524+0000","last_active":"2024-09-16T14:19:04.511421+0000","last_peered":"2024-09-16T14:19:04.511421+0000","last_clean":"2024-09-16T14:19:04.511421+0000","last_became_active":"2024-09-16T14:17:53.481229+0000","last_became_peered":"2024-09-16T14:17:53.481229+0000","last_unstale":"2024-09-16T14:19:04.511421+0000","last_undegraded":"2024-09-16T14:19:04.511421+0000","last_fullsized":"2024-09-16T14:19:04.511421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:04.183728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776620+0000","last_change":"2024-09-16T14:17:51.393181+0000","last_active":"2024-09-16T14:18:22.776620+0000","last_peered":"2024-09-16T14:18:22.776620+0000","last_clean":"2024-09-16T14:18:22.776620+0000","last_became_active":"2024-09-16T14:17:51.392893+0000","last_became_peered":"2024-09-16T14:17:51.392893+0000","last_unstale":"2024-09-16T14:18:22.776620+0000","last_undegraded":"2024-09-16T14:18:22.776620+0000","last_fullsized":"2024-09-16T14:18:22.776620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:34.696377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777870+0000","last_change":"2024-09-16T14:17:55.496800+0000","last_active":"2024-09-16T14:18:22.777870+0000","last_peered":"2024-09-16T14:18:22.777870+0000","last_clean":"2024-09-16T14:18:22.777870+0000","last_became_active":"2024-09-16T14:17:55.496100+0000","last_became_peered":"2024-09-16T14:17:55.496100+0000","last_unstale":"2024-09-16T14:18:22.777870+0000","last_undegraded":"2024-09-16T14:18:22.777870+0000","last_fullsized":"2024-09-16T14:18:22.777870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:34:39.524984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772557+0000","last_change":"2024-09-16T14:17:57.515785+0000","last_active":"2024-09-16T14:18:22.772557+0000","last_peered":"2024-09-16T14:18:22.772557+0000","last_clean":"2024-09-16T14:18:22.772557+0000","last_became_active":"2024-09-16T14:17:57.515035+0000","last_became_peered":"2024-09-16T14:17:57.515035+0000","last_unstale":"2024-09-16T14:18:22.772557+0000","last_undegraded":"2024-09-16T14:18:22.772557+0000","last_fullsized":"2024-09-16T14:18:22.772557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:28.487955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:23.128445+0000","last_change":"2024-09-16T14:18:22.787419+0000","last_active":"2024-09-16T14:18:23.128445+0000","last_peered":"2024-09-16T14:18:23.128445+0000","last_clean":"2024-09-16T14:18:23.128445+0000","last_became_active":"2024-09-16T14:18:22.779558+0000","last_became_peered":"2024-09-16T14:18:22.779558+0000","last_unstale":"2024-09-16T14:18:23.128445+0000","last_undegraded":"2024-09-16T14:18:23.128445+0000","last_fullsized":"2024-09-16T14:18:23.128445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:58.019132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":101,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:08.885266+0000","last_change":"2024-09-16T14:17:55.483857+0000","last_active":"2024-09-16T14:19:08.885266+0000","last_peered":"2024-09-16T14:19:08.885266+0000","last_clean":"2024-09-16T14:19:08.885266+0000","last_became_active":"2024-09-16T14:17:51.399163+0000","last_became_peered":"2024-09-16T14:17:51.399163+0000","last_unstale":"2024-09-16T14:19:08.885266+0000","last_undegraded":"2024-09-16T14:19:08.885266+0000","last_fullsized":"2024-09-16T14:19:08.885266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:44.432547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035163099999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":72,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"61'1","reported_seq":26,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776000+0000","last_change":"2024-09-16T14:17:51.404809+0000","last_active":"2024-09-16T14:18:22.776000+0000","last_peered":"2024-09-16T14:18:22.776000+0000","last_clean":"2024-09-16T14:18:22.776000+0000","last_became_active":"2024-09-16T14:17:51.404285+0000","last_became_peered":"2024-09-16T14:17:51.404285+0000","last_unstale":"2024-09-16T14:18:22.776000+0000","last_undegraded":"2024-09-16T14:18:22.776000+0000","last_fullsized":"2024-09-16T14:18:22.776000+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:12:30.629923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776733+0000","last_change":"2024-09-16T14:17:55.499744+0000","last_active":"2024-09-16T14:18:22.776733+0000","last_peered":"2024-09-16T14:18:22.776733+0000","last_clean":"2024-09-16T14:18:22.776733+0000","last_became_active":"2024-09-16T14:17:55.499432+0000","last_became_peered":"2024-09-16T14:17:55.499432+0000","last_unstale":"2024-09-16T14:18:22.776733+0000","last_undegraded":"2024-09-16T14:18:22.776733+0000","last_fullsized":"2024-09-16T14:18:22.776733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:24.141736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"68'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928279+0000","last_change":"2024-09-16T14:17:57.711153+0000","last_active":"2024-09-16T14:18:22.928279+0000","last_peered":"2024-09-16T14:18:22.928279+0000","last_clean":"2024-09-16T14:18:22.928279+0000","last_became_active":"2024-09-16T14:17:57.710876+0000","last_became_peered":"2024-09-16T14:17:57.710876+0000","last_unstale":"2024-09-16T14:18:22.928279+0000","last_undegraded":"2024-09-16T14:18:22.928279+0000","last_fullsized":"2024-09-16T14:18:22.928279+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:22.083959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":37,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776755+0000","last_change":"2024-09-16T14:17:53.477253+0000","last_active":"2024-09-16T14:18:22.776755+0000","last_peered":"2024-09-16T14:18:22.776755+0000","last_clean":"2024-09-16T14:18:22.776755+0000","last_became_active":"2024-09-16T14:17:53.476948+0000","last_became_peered":"2024-09-16T14:17:53.476948+0000","last_unstale":"2024-09-16T14:18:22.776755+0000","last_undegraded":"2024-09-16T14:18:22.776755+0000","last_fullsized":"2024-09-16T14:18:22.776755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:29.443726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":27,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777589+0000","last_change":"2024-09-16T14:17:55.488175+0000","last_active":"2024-09-16T14:18:22.777589+0000","last_peered":"2024-09-16T14:18:22.777589+0000","last_clean":"2024-09-16T14:18:22.777589+0000","last_became_active":"2024-09-16T14:17:51.396572+0000","last_became_peered":"2024-09-16T14:17:51.396572+0000","last_unstale":"2024-09-16T14:18:22.777589+0000","last_undegraded":"2024-09-16T14:18:22.777589+0000","last_fullsized":"2024-09-16T14:18:22.777589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:50:22.697464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049759299999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928238+0000","last_change":"2024-09-16T14:17:51.400300+0000","last_active":"2024-09-16T14:18:22.928238+0000","last_peered":"2024-09-16T14:18:22.928238+0000","last_clean":"2024-09-16T14:18:22.928238+0000","last_became_active":"2024-09-16T14:17:51.399990+0000","last_became_peered":"2024-09-16T14:17:51.399990+0000","last_unstale":"2024-09-16T14:18:22.928238+0000","last_undegraded":"2024-09-16T14:18:22.928238+0000","last_fullsized":"2024-09-16T14:18:22.928238+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:36.797120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777231+0000","last_change":"2024-09-16T14:17:55.496197+0000","last_active":"2024-09-16T14:18:22.777231+0000","last_peered":"2024-09-16T14:18:22.777231+0000","last_clean":"2024-09-16T14:18:22.777231+0000","last_became_active":"2024-09-16T14:17:55.495892+0000","last_became_peered":"2024-09-16T14:17:55.495892+0000","last_unstale":"2024-09-16T14:18:22.777231+0000","last_undegraded":"2024-09-16T14:18:22.777231+0000","last_fullsized":"2024-09-16T14:18:22.777231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:04:21.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771829+0000","last_change":"2024-09-16T14:17:57.515933+0000","last_active":"2024-09-16T14:18:22.771829+0000","last_peered":"2024-09-16T14:18:22.771829+0000","last_clean":"2024-09-16T14:18:22.771829+0000","last_became_active":"2024-09-16T14:17:57.515595+0000","last_became_peered":"2024-09-16T14:17:57.515595+0000","last_unstale":"2024-09-16T14:18:22.771829+0000","last_undegraded":"2024-09-16T14:18:22.771829+0000","last_fullsized":"2024-09-16T14:18:22.771829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:56.474292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'14","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928515+0000","last_change":"2024-09-16T14:17:53.850653+0000","last_active":"2024-09-16T14:18:22.928515+0000","last_peered":"2024-09-16T14:18:22.928515+0000","last_clean":"2024-09-16T14:18:22.928515+0000","last_became_active":"2024-09-16T14:17:53.850318+0000","last_became_peered":"2024-09-16T14:17:53.850318+0000","last_unstale":"2024-09-16T14:18:22.928515+0000","last_undegraded":"2024-09-16T14:18:22.928515+0000","last_fullsized":"2024-09-16T14:18:22.928515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:37.630005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771489+0000","last_change":"2024-09-16T14:17:55.481197+0000","last_active":"2024-09-16T14:18:22.771489+0000","last_peered":"2024-09-16T14:18:22.771489+0000","last_clean":"2024-09-16T14:18:22.771489+0000","last_became_active":"2024-09-16T14:17:51.400602+0000","last_became_peered":"2024-09-16T14:17:51.400602+0000","last_unstale":"2024-09-16T14:18:22.771489+0000","last_undegraded":"2024-09-16T14:18:22.771489+0000","last_fullsized":"2024-09-16T14:18:22.771489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:32.518153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00108147,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776767+0000","last_change":"2024-09-16T14:17:51.402806+0000","last_active":"2024-09-16T14:18:22.776767+0000","last_peered":"2024-09-16T14:18:22.776767+0000","last_clean":"2024-09-16T14:18:22.776767+0000","last_became_active":"2024-09-16T14:17:51.402531+0000","last_became_peered":"2024-09-16T14:17:51.402531+0000","last_unstale":"2024-09-16T14:18:22.776767+0000","last_undegraded":"2024-09-16T14:18:22.776767+0000","last_fullsized":"2024-09-16T14:18:22.776767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:30.290994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'188","reported_seq":338,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:18:31.645052+0000","last_change":"2024-09-16T14:17:40.182849+0000","last_active":"2024-09-16T14:18:31.645052+0000","last_peered":"2024-09-16T14:18:31.645052+0000","last_clean":"2024-09-16T14:18:31.645052+0000","last_became_active":"2024-09-16T14:17:40.182404+0000","last_became_peered":"2024-09-16T14:17:40.182404+0000","last_unstale":"2024-09-16T14:18:31.645052+0000","last_undegraded":"2024-09-16T14:18:31.645052+0000","last_fullsized":"2024-09-16T14:18:31.645052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:15:34.108483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:15:34.108483+0000","last_clean_scrub_stamp":"2024-09-16T14:15:34.108483+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:41.781469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928432+0000","last_change":"2024-09-16T14:17:55.499840+0000","last_active":"2024-09-16T14:18:22.928432+0000","last_peered":"2024-09-16T14:18:22.928432+0000","last_clean":"2024-09-16T14:18:22.928432+0000","last_became_active":"2024-09-16T14:17:55.499558+0000","last_became_peered":"2024-09-16T14:17:55.499558+0000","last_unstale":"2024-09-16T14:18:22.928432+0000","last_undegraded":"2024-09-16T14:18:22.928432+0000","last_fullsized":"2024-09-16T14:18:22.928432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:27.103751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777281+0000","last_change":"2024-09-16T14:17:57.509730+0000","last_active":"2024-09-16T14:18:22.777281+0000","last_peered":"2024-09-16T14:18:22.777281+0000","last_clean":"2024-09-16T14:18:22.777281+0000","last_became_active":"2024-09-16T14:17:57.509444+0000","last_became_peered":"2024-09-16T14:17:57.509444+0000","last_unstale":"2024-09-16T14:18:22.777281+0000","last_undegraded":"2024-09-16T14:18:22.777281+0000","last_fullsized":"2024-09-16T14:18:22.777281+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:47.956966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928186+0000","last_change":"2024-09-16T14:17:53.850605+0000","last_active":"2024-09-16T14:18:22.928186+0000","last_peered":"2024-09-16T14:18:22.928186+0000","last_clean":"2024-09-16T14:18:22.928186+0000","last_became_active":"2024-09-16T14:17:53.850276+0000","last_became_peered":"2024-09-16T14:17:53.850276+0000","last_unstale":"2024-09-16T14:18:22.928186+0000","last_undegraded":"2024-09-16T14:18:22.928186+0000","last_fullsized":"2024-09-16T14:18:22.928186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:00.311782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777167+0000","last_change":"2024-09-16T14:17:51.404540+0000","last_active":"2024-09-16T14:18:22.777167+0000","last_peered":"2024-09-16T14:18:22.777167+0000","last_clean":"2024-09-16T14:18:22.777167+0000","last_became_active":"2024-09-16T14:17:51.404124+0000","last_became_peered":"2024-09-16T14:17:51.404124+0000","last_unstale":"2024-09-16T14:18:22.777167+0000","last_undegraded":"2024-09-16T14:18:22.777167+0000","last_fullsized":"2024-09-16T14:18:22.777167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:01.026961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775758+0000","last_change":"2024-09-16T14:17:55.495331+0000","last_active":"2024-09-16T14:18:22.775758+0000","last_peered":"2024-09-16T14:18:22.775758+0000","last_clean":"2024-09-16T14:18:22.775758+0000","last_became_active":"2024-09-16T14:17:55.494932+0000","last_became_peered":"2024-09-16T14:17:55.494932+0000","last_unstale":"2024-09-16T14:18:22.775758+0000","last_undegraded":"2024-09-16T14:18:22.775758+0000","last_fullsized":"2024-09-16T14:18:22.775758+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:45.021548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928337+0000","last_change":"2024-09-16T14:17:57.516188+0000","last_active":"2024-09-16T14:18:22.928337+0000","last_peered":"2024-09-16T14:18:22.928337+0000","last_clean":"2024-09-16T14:18:22.928337+0000","last_became_active":"2024-09-16T14:17:57.515933+0000","last_became_peered":"2024-09-16T14:17:57.515933+0000","last_unstale":"2024-09-16T14:18:22.928337+0000","last_undegraded":"2024-09-16T14:18:22.928337+0000","last_fullsized":"2024-09-16T14:18:22.928337+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:15.551338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.778827+0000","last_change":"2024-09-16T14:17:53.482565+0000","last_active":"2024-09-16T14:18:22.778827+0000","last_peered":"2024-09-16T14:18:22.778827+0000","last_clean":"2024-09-16T14:18:22.778827+0000","last_became_active":"2024-09-16T14:17:53.482294+0000","last_became_peered":"2024-09-16T14:17:53.482294+0000","last_unstale":"2024-09-16T14:18:22.778827+0000","last_undegraded":"2024-09-16T14:18:22.778827+0000","last_fullsized":"2024-09-16T14:18:22.778827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:34.095826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.773041+0000","last_change":"2024-09-16T14:17:51.390862+0000","last_active":"2024-09-16T14:18:22.773041+0000","last_peered":"2024-09-16T14:18:22.773041+0000","last_clean":"2024-09-16T14:18:22.773041+0000","last_became_active":"2024-09-16T14:17:51.390494+0000","last_became_peered":"2024-09-16T14:17:51.390494+0000","last_unstale":"2024-09-16T14:18:22.773041+0000","last_undegraded":"2024-09-16T14:18:22.773041+0000","last_fullsized":"2024-09-16T14:18:22.773041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:53:33.818723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775452+0000","last_change":"2024-09-16T14:17:55.498231+0000","last_active":"2024-09-16T14:18:22.775452+0000","last_peered":"2024-09-16T14:18:22.775452+0000","last_clean":"2024-09-16T14:18:22.775452+0000","last_became_active":"2024-09-16T14:17:55.497835+0000","last_became_peered":"2024-09-16T14:17:55.497835+0000","last_unstale":"2024-09-16T14:18:22.775452+0000","last_undegraded":"2024-09-16T14:18:22.775452+0000","last_fullsized":"2024-09-16T14:18:22.775452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:25.144897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777512+0000","last_change":"2024-09-16T14:17:57.512137+0000","last_active":"2024-09-16T14:18:22.777512+0000","last_peered":"2024-09-16T14:18:22.777512+0000","last_clean":"2024-09-16T14:18:22.777512+0000","last_became_active":"2024-09-16T14:17:57.511800+0000","last_became_peered":"2024-09-16T14:17:57.511800+0000","last_unstale":"2024-09-16T14:18:22.777512+0000","last_undegraded":"2024-09-16T14:18:22.777512+0000","last_fullsized":"2024-09-16T14:18:22.777512+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:27:46.653558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776116+0000","last_change":"2024-09-16T14:17:53.850692+0000","last_active":"2024-09-16T14:18:22.776116+0000","last_peered":"2024-09-16T14:18:22.776116+0000","last_clean":"2024-09-16T14:18:22.776116+0000","last_became_active":"2024-09-16T14:17:53.850259+0000","last_became_peered":"2024-09-16T14:17:53.850259+0000","last_unstale":"2024-09-16T14:18:22.776116+0000","last_undegraded":"2024-09-16T14:18:22.776116+0000","last_fullsized":"2024-09-16T14:18:22.776116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:44.042366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.780581+0000","last_change":"2024-09-16T14:18:22.779939+0000","last_active":"2024-09-16T14:18:22.780581+0000","last_peered":"2024-09-16T14:18:22.780581+0000","last_clean":"2024-09-16T14:18:22.780581+0000","last_became_active":"2024-09-16T14:18:22.779587+0000","last_became_peered":"2024-09-16T14:18:22.779587+0000","last_unstale":"2024-09-16T14:18:22.780581+0000","last_undegraded":"2024-09-16T14:18:22.780581+0000","last_fullsized":"2024-09-16T14:18:22.780581+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:14.945046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511576+0000","last_change":"2024-09-16T14:17:55.496084+0000","last_active":"2024-09-16T14:19:04.511576+0000","last_peered":"2024-09-16T14:19:04.511576+0000","last_clean":"2024-09-16T14:19:04.511576+0000","last_became_active":"2024-09-16T14:17:55.495807+0000","last_became_peered":"2024-09-16T14:17:55.495807+0000","last_unstale":"2024-09-16T14:19:04.511576+0000","last_undegraded":"2024-09-16T14:19:04.511576+0000","last_fullsized":"2024-09-16T14:19:04.511576+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:44.423911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776349+0000","last_change":"2024-09-16T14:17:57.711145+0000","last_active":"2024-09-16T14:18:22.776349+0000","last_peered":"2024-09-16T14:18:22.776349+0000","last_clean":"2024-09-16T14:18:22.776349+0000","last_became_active":"2024-09-16T14:17:57.710861+0000","last_became_peered":"2024-09-16T14:17:57.710861+0000","last_unstale":"2024-09-16T14:18:22.776349+0000","last_undegraded":"2024-09-16T14:18:22.776349+0000","last_fullsized":"2024-09-16T14:18:22.776349+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:52.948185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771401+0000","last_change":"2024-09-16T14:17:53.477373+0000","last_active":"2024-09-16T14:18:22.771401+0000","last_peered":"2024-09-16T14:18:22.771401+0000","last_clean":"2024-09-16T14:18:22.771401+0000","last_became_active":"2024-09-16T14:17:53.477051+0000","last_became_peered":"2024-09-16T14:17:53.477051+0000","last_unstale":"2024-09-16T14:18:22.771401+0000","last_undegraded":"2024-09-16T14:18:22.771401+0000","last_fullsized":"2024-09-16T14:18:22.771401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:00.948128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772201+0000","last_change":"2024-09-16T14:17:51.395616+0000","last_active":"2024-09-16T14:18:22.772201+0000","last_peered":"2024-09-16T14:18:22.772201+0000","last_clean":"2024-09-16T14:18:22.772201+0000","last_became_active":"2024-09-16T14:17:51.395331+0000","last_became_peered":"2024-09-16T14:17:51.395331+0000","last_unstale":"2024-09-16T14:18:22.772201+0000","last_undegraded":"2024-09-16T14:18:22.772201+0000","last_fullsized":"2024-09-16T14:18:22.772201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:45.937702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775309+0000","last_change":"2024-09-16T14:17:55.491163+0000","last_active":"2024-09-16T14:18:22.775309+0000","last_peered":"2024-09-16T14:18:22.775309+0000","last_clean":"2024-09-16T14:18:22.775309+0000","last_became_active":"2024-09-16T14:17:55.490860+0000","last_became_peered":"2024-09-16T14:17:55.490860+0000","last_unstale":"2024-09-16T14:18:22.775309+0000","last_undegraded":"2024-09-16T14:18:22.775309+0000","last_fullsized":"2024-09-16T14:18:22.775309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:57.020362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771496+0000","last_change":"2024-09-16T14:17:57.514127+0000","last_active":"2024-09-16T14:18:22.771496+0000","last_peered":"2024-09-16T14:18:22.771496+0000","last_clean":"2024-09-16T14:18:22.771496+0000","last_became_active":"2024-09-16T14:17:57.513834+0000","last_became_peered":"2024-09-16T14:17:57.513834+0000","last_unstale":"2024-09-16T14:18:22.771496+0000","last_undegraded":"2024-09-16T14:18:22.771496+0000","last_fullsized":"2024-09-16T14:18:22.771496+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:34.242501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776653+0000","last_change":"2024-09-16T14:17:53.851506+0000","last_active":"2024-09-16T14:18:22.776653+0000","last_peered":"2024-09-16T14:18:22.776653+0000","last_clean":"2024-09-16T14:18:22.776653+0000","last_became_active":"2024-09-16T14:17:53.850278+0000","last_became_peered":"2024-09-16T14:17:53.850278+0000","last_unstale":"2024-09-16T14:18:22.776653+0000","last_undegraded":"2024-09-16T14:18:22.776653+0000","last_fullsized":"2024-09-16T14:18:22.776653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:30.592547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776043+0000","last_change":"2024-09-16T14:17:51.405086+0000","last_active":"2024-09-16T14:18:22.776043+0000","last_peered":"2024-09-16T14:18:22.776043+0000","last_clean":"2024-09-16T14:18:22.776043+0000","last_became_active":"2024-09-16T14:17:51.404619+0000","last_became_peered":"2024-09-16T14:17:51.404619+0000","last_unstale":"2024-09-16T14:18:22.776043+0000","last_undegraded":"2024-09-16T14:18:22.776043+0000","last_fullsized":"2024-09-16T14:18:22.776043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:17.279280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772132+0000","last_change":"2024-09-16T14:17:55.491871+0000","last_active":"2024-09-16T14:18:22.772132+0000","last_peered":"2024-09-16T14:18:22.772132+0000","last_clean":"2024-09-16T14:18:22.772132+0000","last_became_active":"2024-09-16T14:17:55.491556+0000","last_became_peered":"2024-09-16T14:17:55.491556+0000","last_unstale":"2024-09-16T14:18:22.772132+0000","last_undegraded":"2024-09-16T14:18:22.772132+0000","last_fullsized":"2024-09-16T14:18:22.772132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:04.892734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.775635+0000","last_change":"2024-09-16T14:17:57.514463+0000","last_active":"2024-09-16T14:18:22.775635+0000","last_peered":"2024-09-16T14:18:22.775635+0000","last_clean":"2024-09-16T14:18:22.775635+0000","last_became_active":"2024-09-16T14:17:57.514043+0000","last_became_peered":"2024-09-16T14:17:57.514043+0000","last_unstale":"2024-09-16T14:18:22.775635+0000","last_undegraded":"2024-09-16T14:18:22.775635+0000","last_fullsized":"2024-09-16T14:18:22.775635+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:09.920931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.927926+0000","last_change":"2024-09-16T14:17:51.395406+0000","last_active":"2024-09-16T14:18:22.927926+0000","last_peered":"2024-09-16T14:18:22.927926+0000","last_clean":"2024-09-16T14:18:22.927926+0000","last_became_active":"2024-09-16T14:17:51.395077+0000","last_became_peered":"2024-09-16T14:17:51.395077+0000","last_unstale":"2024-09-16T14:18:22.927926+0000","last_undegraded":"2024-09-16T14:18:22.927926+0000","last_fullsized":"2024-09-16T14:18:22.927926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:03.552272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":28,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772280+0000","last_change":"2024-09-16T14:17:53.851047+0000","last_active":"2024-09-16T14:18:22.772280+0000","last_peered":"2024-09-16T14:18:22.772280+0000","last_clean":"2024-09-16T14:18:22.772280+0000","last_became_active":"2024-09-16T14:17:53.850750+0000","last_became_peered":"2024-09-16T14:17:53.850750+0000","last_unstale":"2024-09-16T14:18:22.772280+0000","last_undegraded":"2024-09-16T14:18:22.772280+0000","last_fullsized":"2024-09-16T14:18:22.772280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:00.046289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776999+0000","last_change":"2024-09-16T14:17:55.493178+0000","last_active":"2024-09-16T14:18:22.776999+0000","last_peered":"2024-09-16T14:18:22.776999+0000","last_clean":"2024-09-16T14:18:22.776999+0000","last_became_active":"2024-09-16T14:17:55.492894+0000","last_became_peered":"2024-09-16T14:17:55.492894+0000","last_unstale":"2024-09-16T14:18:22.776999+0000","last_undegraded":"2024-09-16T14:18:22.776999+0000","last_fullsized":"2024-09-16T14:18:22.776999+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:28:28.920343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928193+0000","last_change":"2024-09-16T14:17:57.712055+0000","last_active":"2024-09-16T14:18:22.928193+0000","last_peered":"2024-09-16T14:18:22.928193+0000","last_clean":"2024-09-16T14:18:22.928193+0000","last_became_active":"2024-09-16T14:17:57.711795+0000","last_became_peered":"2024-09-16T14:17:57.711795+0000","last_unstale":"2024-09-16T14:18:22.928193+0000","last_undegraded":"2024-09-16T14:18:22.928193+0000","last_fullsized":"2024-09-16T14:18:22.928193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:20.713477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772147+0000","last_change":"2024-09-16T14:17:53.851736+0000","last_active":"2024-09-16T14:18:22.772147+0000","last_peered":"2024-09-16T14:18:22.772147+0000","last_clean":"2024-09-16T14:18:22.772147+0000","last_became_active":"2024-09-16T14:17:53.850102+0000","last_became_peered":"2024-09-16T14:17:53.850102+0000","last_unstale":"2024-09-16T14:18:22.772147+0000","last_undegraded":"2024-09-16T14:18:22.772147+0000","last_fullsized":"2024-09-16T14:18:22.772147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:34:29.765267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776576+0000","last_change":"2024-09-16T14:17:51.397551+0000","last_active":"2024-09-16T14:18:22.776576+0000","last_peered":"2024-09-16T14:18:22.776576+0000","last_clean":"2024-09-16T14:18:22.776576+0000","last_became_active":"2024-09-16T14:17:51.397116+0000","last_became_peered":"2024-09-16T14:17:51.397116+0000","last_unstale":"2024-09-16T14:18:22.776576+0000","last_undegraded":"2024-09-16T14:18:22.776576+0000","last_fullsized":"2024-09-16T14:18:22.776576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:16.588915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511619+0000","last_change":"2024-09-16T14:17:55.494870+0000","last_active":"2024-09-16T14:19:04.511619+0000","last_peered":"2024-09-16T14:19:04.511619+0000","last_clean":"2024-09-16T14:19:04.511619+0000","last_became_active":"2024-09-16T14:17:55.494503+0000","last_became_peered":"2024-09-16T14:17:55.494503+0000","last_unstale":"2024-09-16T14:19:04.511619+0000","last_undegraded":"2024-09-16T14:19:04.511619+0000","last_fullsized":"2024-09-16T14:19:04.511619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:13.273987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776329+0000","last_change":"2024-09-16T14:17:57.512950+0000","last_active":"2024-09-16T14:18:22.776329+0000","last_peered":"2024-09-16T14:18:22.776329+0000","last_clean":"2024-09-16T14:18:22.776329+0000","last_became_active":"2024-09-16T14:17:57.512644+0000","last_became_peered":"2024-09-16T14:17:57.512644+0000","last_unstale":"2024-09-16T14:18:22.776329+0000","last_undegraded":"2024-09-16T14:18:22.776329+0000","last_fullsized":"2024-09-16T14:18:22.776329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:25:55.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776866+0000","last_change":"2024-09-16T14:17:53.850629+0000","last_active":"2024-09-16T14:18:22.776866+0000","last_peered":"2024-09-16T14:18:22.776866+0000","last_clean":"2024-09-16T14:18:22.776866+0000","last_became_active":"2024-09-16T14:17:53.850293+0000","last_became_peered":"2024-09-16T14:17:53.850293+0000","last_unstale":"2024-09-16T14:18:22.776866+0000","last_undegraded":"2024-09-16T14:18:22.776866+0000","last_fullsized":"2024-09-16T14:18:22.776866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:33.472563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.779017+0000","last_change":"2024-09-16T14:17:51.391840+0000","last_active":"2024-09-16T14:18:22.779017+0000","last_peered":"2024-09-16T14:18:22.779017+0000","last_clean":"2024-09-16T14:18:22.779017+0000","last_became_active":"2024-09-16T14:17:51.391536+0000","last_became_peered":"2024-09-16T14:17:51.391536+0000","last_unstale":"2024-09-16T14:18:22.779017+0000","last_undegraded":"2024-09-16T14:18:22.779017+0000","last_fullsized":"2024-09-16T14:18:22.779017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:49:50.059590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511444+0000","last_change":"2024-09-16T14:17:55.490014+0000","last_active":"2024-09-16T14:19:04.511444+0000","last_peered":"2024-09-16T14:19:04.511444+0000","last_clean":"2024-09-16T14:19:04.511444+0000","last_became_active":"2024-09-16T14:17:55.489674+0000","last_became_peered":"2024-09-16T14:17:55.489674+0000","last_unstale":"2024-09-16T14:19:04.511444+0000","last_undegraded":"2024-09-16T14:19:04.511444+0000","last_fullsized":"2024-09-16T14:19:04.511444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:12:49.143663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772057+0000","last_change":"2024-09-16T14:17:57.507197+0000","last_active":"2024-09-16T14:18:22.772057+0000","last_peered":"2024-09-16T14:18:22.772057+0000","last_clean":"2024-09-16T14:18:22.772057+0000","last_became_active":"2024-09-16T14:17:57.506894+0000","last_became_peered":"2024-09-16T14:17:57.506894+0000","last_unstale":"2024-09-16T14:18:22.772057+0000","last_undegraded":"2024-09-16T14:18:22.772057+0000","last_fullsized":"2024-09-16T14:18:22.772057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:29.164369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772064+0000","last_change":"2024-09-16T14:17:53.482350+0000","last_active":"2024-09-16T14:18:22.772064+0000","last_peered":"2024-09-16T14:18:22.772064+0000","last_clean":"2024-09-16T14:18:22.772064+0000","last_became_active":"2024-09-16T14:17:53.482009+0000","last_became_peered":"2024-09-16T14:17:53.482009+0000","last_unstale":"2024-09-16T14:18:22.772064+0000","last_undegraded":"2024-09-16T14:18:22.772064+0000","last_fullsized":"2024-09-16T14:18:22.772064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:28.565243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928359+0000","last_change":"2024-09-16T14:17:51.394421+0000","last_active":"2024-09-16T14:18:22.928359+0000","last_peered":"2024-09-16T14:18:22.928359+0000","last_clean":"2024-09-16T14:18:22.928359+0000","last_became_active":"2024-09-16T14:17:51.394137+0000","last_became_peered":"2024-09-16T14:17:51.394137+0000","last_unstale":"2024-09-16T14:18:22.928359+0000","last_undegraded":"2024-09-16T14:18:22.928359+0000","last_fullsized":"2024-09-16T14:18:22.928359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:54.060065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771641+0000","last_change":"2024-09-16T14:17:55.493439+0000","last_active":"2024-09-16T14:18:22.771641+0000","last_peered":"2024-09-16T14:18:22.771641+0000","last_clean":"2024-09-16T14:18:22.771641+0000","last_became_active":"2024-09-16T14:17:55.493140+0000","last_became_peered":"2024-09-16T14:17:55.493140+0000","last_unstale":"2024-09-16T14:18:22.771641+0000","last_undegraded":"2024-09-16T14:18:22.771641+0000","last_fullsized":"2024-09-16T14:18:22.771641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:28.917395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776282+0000","last_change":"2024-09-16T14:17:57.515187+0000","last_active":"2024-09-16T14:18:22.776282+0000","last_peered":"2024-09-16T14:18:22.776282+0000","last_clean":"2024-09-16T14:18:22.776282+0000","last_became_active":"2024-09-16T14:17:57.514892+0000","last_became_peered":"2024-09-16T14:17:57.514892+0000","last_unstale":"2024-09-16T14:18:22.776282+0000","last_undegraded":"2024-09-16T14:18:22.776282+0000","last_fullsized":"2024-09-16T14:18:22.776282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:28.297737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771540+0000","last_change":"2024-09-16T14:17:53.480543+0000","last_active":"2024-09-16T14:18:22.771540+0000","last_peered":"2024-09-16T14:18:22.771540+0000","last_clean":"2024-09-16T14:18:22.771540+0000","last_became_active":"2024-09-16T14:17:53.480248+0000","last_became_peered":"2024-09-16T14:17:53.480248+0000","last_unstale":"2024-09-16T14:18:22.771540+0000","last_undegraded":"2024-09-16T14:18:22.771540+0000","last_fullsized":"2024-09-16T14:18:22.771540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:40.337291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772963+0000","last_change":"2024-09-16T14:17:51.392587+0000","last_active":"2024-09-16T14:18:22.772963+0000","last_peered":"2024-09-16T14:18:22.772963+0000","last_clean":"2024-09-16T14:18:22.772963+0000","last_became_active":"2024-09-16T14:17:51.392382+0000","last_became_peered":"2024-09-16T14:17:51.392382+0000","last_unstale":"2024-09-16T14:18:22.772963+0000","last_undegraded":"2024-09-16T14:18:22.772963+0000","last_fullsized":"2024-09-16T14:18:22.772963+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:53.476072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771917+0000","last_change":"2024-09-16T14:17:55.493497+0000","last_active":"2024-09-16T14:18:22.771917+0000","last_peered":"2024-09-16T14:18:22.771917+0000","last_clean":"2024-09-16T14:18:22.771917+0000","last_became_active":"2024-09-16T14:17:55.493233+0000","last_became_peered":"2024-09-16T14:17:55.493233+0000","last_unstale":"2024-09-16T14:18:22.771917+0000","last_undegraded":"2024-09-16T14:18:22.771917+0000","last_fullsized":"2024-09-16T14:18:22.771917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:49.274458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771215+0000","last_change":"2024-09-16T14:17:57.509385+0000","last_active":"2024-09-16T14:18:22.771215+0000","last_peered":"2024-09-16T14:18:22.771215+0000","last_clean":"2024-09-16T14:18:22.771215+0000","last_became_active":"2024-09-16T14:17:57.509043+0000","last_became_peered":"2024-09-16T14:17:57.509043+0000","last_unstale":"2024-09-16T14:18:22.771215+0000","last_undegraded":"2024-09-16T14:18:22.771215+0000","last_fullsized":"2024-09-16T14:18:22.771215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:40.600301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771030+0000","last_change":"2024-09-16T14:17:53.485198+0000","last_active":"2024-09-16T14:18:22.771030+0000","last_peered":"2024-09-16T14:18:22.771030+0000","last_clean":"2024-09-16T14:18:22.771030+0000","last_became_active":"2024-09-16T14:17:53.484733+0000","last_became_peered":"2024-09-16T14:17:53.484733+0000","last_unstale":"2024-09-16T14:18:22.771030+0000","last_undegraded":"2024-09-16T14:18:22.771030+0000","last_fullsized":"2024-09-16T14:18:22.771030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:36.276380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.777044+0000","last_change":"2024-09-16T14:17:51.399661+0000","last_active":"2024-09-16T14:18:22.777044+0000","last_peered":"2024-09-16T14:18:22.777044+0000","last_clean":"2024-09-16T14:18:22.777044+0000","last_became_active":"2024-09-16T14:17:51.399337+0000","last_became_peered":"2024-09-16T14:17:51.399337+0000","last_unstale":"2024-09-16T14:18:22.777044+0000","last_undegraded":"2024-09-16T14:18:22.777044+0000","last_fullsized":"2024-09-16T14:18:22.777044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:29.495774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.927728+0000","last_change":"2024-09-16T14:17:55.496036+0000","last_active":"2024-09-16T14:18:22.927728+0000","last_peered":"2024-09-16T14:18:22.927728+0000","last_clean":"2024-09-16T14:18:22.927728+0000","last_became_active":"2024-09-16T14:17:55.495687+0000","last_became_peered":"2024-09-16T14:17:55.495687+0000","last_unstale":"2024-09-16T14:18:22.927728+0000","last_undegraded":"2024-09-16T14:18:22.927728+0000","last_fullsized":"2024-09-16T14:18:22.927728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:44:17.423745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928152+0000","last_change":"2024-09-16T14:17:57.517006+0000","last_active":"2024-09-16T14:18:22.928152+0000","last_peered":"2024-09-16T14:18:22.928152+0000","last_clean":"2024-09-16T14:18:22.928152+0000","last_became_active":"2024-09-16T14:17:57.516460+0000","last_became_peered":"2024-09-16T14:17:57.516460+0000","last_unstale":"2024-09-16T14:18:22.928152+0000","last_undegraded":"2024-09-16T14:18:22.928152+0000","last_fullsized":"2024-09-16T14:18:22.928152+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:22.909755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771437+0000","last_change":"2024-09-16T14:17:53.850843+0000","last_active":"2024-09-16T14:18:22.771437+0000","last_peered":"2024-09-16T14:18:22.771437+0000","last_clean":"2024-09-16T14:18:22.771437+0000","last_became_active":"2024-09-16T14:17:53.850087+0000","last_became_peered":"2024-09-16T14:17:53.850087+0000","last_unstale":"2024-09-16T14:18:22.771437+0000","last_undegraded":"2024-09-16T14:18:22.771437+0000","last_fullsized":"2024-09-16T14:18:22.771437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:00.822052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928633+0000","last_change":"2024-09-16T14:17:51.401102+0000","last_active":"2024-09-16T14:18:22.928633+0000","last_peered":"2024-09-16T14:18:22.928633+0000","last_clean":"2024-09-16T14:18:22.928633+0000","last_became_active":"2024-09-16T14:17:51.400715+0000","last_became_peered":"2024-09-16T14:17:51.400715+0000","last_unstale":"2024-09-16T14:18:22.928633+0000","last_undegraded":"2024-09-16T14:18:22.928633+0000","last_fullsized":"2024-09-16T14:18:22.928633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:31.456651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.927845+0000","last_change":"2024-09-16T14:17:55.498006+0000","last_active":"2024-09-16T14:18:22.927845+0000","last_peered":"2024-09-16T14:18:22.927845+0000","last_clean":"2024-09-16T14:18:22.927845+0000","last_became_active":"2024-09-16T14:17:55.497676+0000","last_became_peered":"2024-09-16T14:17:55.497676+0000","last_unstale":"2024-09-16T14:18:22.927845+0000","last_undegraded":"2024-09-16T14:18:22.927845+0000","last_fullsized":"2024-09-16T14:18:22.927845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:43.597566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771419+0000","last_change":"2024-09-16T14:17:57.712962+0000","last_active":"2024-09-16T14:18:22.771419+0000","last_peered":"2024-09-16T14:18:22.771419+0000","last_clean":"2024-09-16T14:18:22.771419+0000","last_became_active":"2024-09-16T14:17:57.712636+0000","last_became_peered":"2024-09-16T14:17:57.712636+0000","last_unstale":"2024-09-16T14:18:22.771419+0000","last_undegraded":"2024-09-16T14:18:22.771419+0000","last_fullsized":"2024-09-16T14:18:22.771419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:43.967118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928359+0000","last_change":"2024-09-16T14:17:57.516711+0000","last_active":"2024-09-16T14:18:22.928359+0000","last_peered":"2024-09-16T14:18:22.928359+0000","last_clean":"2024-09-16T14:18:22.928359+0000","last_became_active":"2024-09-16T14:17:57.516232+0000","last_became_peered":"2024-09-16T14:17:57.516232+0000","last_unstale":"2024-09-16T14:18:22.928359+0000","last_undegraded":"2024-09-16T14:18:22.928359+0000","last_fullsized":"2024-09-16T14:18:22.928359+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:58.434521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"61'1","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776170+0000","last_change":"2024-09-16T14:17:51.396186+0000","last_active":"2024-09-16T14:18:22.776170+0000","last_peered":"2024-09-16T14:18:22.776170+0000","last_clean":"2024-09-16T14:18:22.776170+0000","last_became_active":"2024-09-16T14:17:51.395926+0000","last_became_peered":"2024-09-16T14:17:51.395926+0000","last_unstale":"2024-09-16T14:18:22.776170+0000","last_undegraded":"2024-09-16T14:18:22.776170+0000","last_fullsized":"2024-09-16T14:18:22.776170+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:02.599865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776565+0000","last_change":"2024-09-16T14:17:53.851170+0000","last_active":"2024-09-16T14:18:22.776565+0000","last_peered":"2024-09-16T14:18:22.776565+0000","last_clean":"2024-09-16T14:18:22.776565+0000","last_became_active":"2024-09-16T14:17:53.850880+0000","last_became_peered":"2024-09-16T14:17:53.850880+0000","last_unstale":"2024-09-16T14:18:22.776565+0000","last_undegraded":"2024-09-16T14:18:22.776565+0000","last_fullsized":"2024-09-16T14:18:22.776565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:13.650295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T14:19:04.511594+0000","last_change":"2024-09-16T14:17:55.496464+0000","last_active":"2024-09-16T14:19:04.511594+0000","last_peered":"2024-09-16T14:19:04.511594+0000","last_clean":"2024-09-16T14:19:04.511594+0000","last_became_active":"2024-09-16T14:17:55.496171+0000","last_became_peered":"2024-09-16T14:17:55.496171+0000","last_unstale":"2024-09-16T14:19:04.511594+0000","last_undegraded":"2024-09-16T14:19:04.511594+0000","last_fullsized":"2024-09-16T14:19:04.511594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:06.757009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772410+0000","last_change":"2024-09-16T14:17:57.516003+0000","last_active":"2024-09-16T14:18:22.772410+0000","last_peered":"2024-09-16T14:18:22.772410+0000","last_clean":"2024-09-16T14:18:22.772410+0000","last_became_active":"2024-09-16T14:17:57.515678+0000","last_became_peered":"2024-09-16T14:17:57.515678+0000","last_unstale":"2024-09-16T14:18:22.772410+0000","last_undegraded":"2024-09-16T14:18:22.772410+0000","last_fullsized":"2024-09-16T14:18:22.772410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:04:36.333359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771661+0000","last_change":"2024-09-16T14:17:51.403154+0000","last_active":"2024-09-16T14:18:22.771661+0000","last_peered":"2024-09-16T14:18:22.771661+0000","last_clean":"2024-09-16T14:18:22.771661+0000","last_became_active":"2024-09-16T14:17:51.402356+0000","last_became_peered":"2024-09-16T14:17:51.402356+0000","last_unstale":"2024-09-16T14:18:22.771661+0000","last_undegraded":"2024-09-16T14:18:22.771661+0000","last_fullsized":"2024-09-16T14:18:22.771661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:34.757407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928602+0000","last_change":"2024-09-16T14:17:53.479115+0000","last_active":"2024-09-16T14:18:22.928602+0000","last_peered":"2024-09-16T14:18:22.928602+0000","last_clean":"2024-09-16T14:18:22.928602+0000","last_became_active":"2024-09-16T14:17:53.478772+0000","last_became_peered":"2024-09-16T14:17:53.478772+0000","last_unstale":"2024-09-16T14:18:22.928602+0000","last_undegraded":"2024-09-16T14:18:22.928602+0000","last_fullsized":"2024-09-16T14:18:22.928602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:52:44.969859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.776416+0000","last_change":"2024-09-16T14:17:55.500316+0000","last_active":"2024-09-16T14:18:22.776416+0000","last_peered":"2024-09-16T14:18:22.776416+0000","last_clean":"2024-09-16T14:18:22.776416+0000","last_became_active":"2024-09-16T14:17:55.499990+0000","last_became_peered":"2024-09-16T14:17:55.499990+0000","last_unstale":"2024-09-16T14:18:22.776416+0000","last_undegraded":"2024-09-16T14:18:22.776416+0000","last_fullsized":"2024-09-16T14:18:22.776416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:19.118263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772009+0000","last_change":"2024-09-16T14:17:57.712247+0000","last_active":"2024-09-16T14:18:22.772009+0000","last_peered":"2024-09-16T14:18:22.772009+0000","last_clean":"2024-09-16T14:18:22.772009+0000","last_became_active":"2024-09-16T14:17:57.711812+0000","last_became_peered":"2024-09-16T14:17:57.711812+0000","last_unstale":"2024-09-16T14:18:22.772009+0000","last_undegraded":"2024-09-16T14:18:22.772009+0000","last_fullsized":"2024-09-16T14:18:22.772009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:40:14.572595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.928254+0000","last_change":"2024-09-16T14:17:51.395020+0000","last_active":"2024-09-16T14:18:22.928254+0000","last_peered":"2024-09-16T14:18:22.928254+0000","last_clean":"2024-09-16T14:18:22.928254+0000","last_became_active":"2024-09-16T14:17:51.394737+0000","last_became_peered":"2024-09-16T14:17:51.394737+0000","last_unstale":"2024-09-16T14:18:22.928254+0000","last_undegraded":"2024-09-16T14:18:22.928254+0000","last_fullsized":"2024-09-16T14:18:22.928254+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:01.560498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":69,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.795040+0000","last_change":"2024-09-16T14:18:22.794721+0000","last_active":"2024-09-16T14:18:22.795040+0000","last_peered":"2024-09-16T14:18:22.795040+0000","last_clean":"2024-09-16T14:18:22.795040+0000","last_became_active":"2024-09-16T14:18:22.780180+0000","last_became_peered":"2024-09-16T14:18:22.780180+0000","last_unstale":"2024-09-16T14:18:22.795040+0000","last_undegraded":"2024-09-16T14:18:22.795040+0000","last_fullsized":"2024-09-16T14:18:22.795040+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:19.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772661+0000","last_change":"2024-09-16T14:17:55.497763+0000","last_active":"2024-09-16T14:18:22.772661+0000","last_peered":"2024-09-16T14:18:22.772661+0000","last_clean":"2024-09-16T14:18:22.772661+0000","last_became_active":"2024-09-16T14:17:55.497474+0000","last_became_peered":"2024-09-16T14:17:55.497474+0000","last_unstale":"2024-09-16T14:18:22.772661+0000","last_undegraded":"2024-09-16T14:18:22.772661+0000","last_fullsized":"2024-09-16T14:18:22.772661+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:06.893530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.772059+0000","last_change":"2024-09-16T14:17:57.711982+0000","last_active":"2024-09-16T14:18:22.772059+0000","last_peered":"2024-09-16T14:18:22.772059+0000","last_clean":"2024-09-16T14:18:22.772059+0000","last_became_active":"2024-09-16T14:17:57.711633+0000","last_became_peered":"2024-09-16T14:17:57.711633+0000","last_unstale":"2024-09-16T14:18:22.772059+0000","last_undegraded":"2024-09-16T14:18:22.772059+0000","last_fullsized":"2024-09-16T14:18:22.772059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:56.488603+0000","last_clean_scrub_stamp":"2024-09-16T14:17:56.488603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:44.334874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771818+0000","last_change":"2024-09-16T14:17:51.398026+0000","last_active":"2024-09-16T14:18:22.771818+0000","last_peered":"2024-09-16T14:18:22.771818+0000","last_clean":"2024-09-16T14:18:22.771818+0000","last_became_active":"2024-09-16T14:17:51.397755+0000","last_became_peered":"2024-09-16T14:17:51.397755+0000","last_unstale":"2024-09-16T14:18:22.771818+0000","last_undegraded":"2024-09-16T14:18:22.771818+0000","last_fullsized":"2024-09-16T14:18:22.771818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:50.373945+0000","last_clean_scrub_stamp":"2024-09-16T14:17:50.373945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:02.992907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.787594+0000","last_change":"2024-09-16T14:18:22.787153+0000","last_active":"2024-09-16T14:18:22.787594+0000","last_peered":"2024-09-16T14:18:22.787594+0000","last_clean":"2024-09-16T14:18:22.787594+0000","last_became_active":"2024-09-16T14:18:22.778551+0000","last_became_peered":"2024-09-16T14:18:22.778551+0000","last_unstale":"2024-09-16T14:18:22.787594+0000","last_undegraded":"2024-09-16T14:18:22.787594+0000","last_fullsized":"2024-09-16T14:18:22.787594+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:52.451667+0000","last_clean_scrub_stamp":"2024-09-16T14:17:52.451667+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:12.189019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T14:18:22.771614+0000","last_change":"2024-09-16T14:17:55.493044+0000","last_active":"2024-09-16T14:18:22.771614+0000","last_peered":"2024-09-16T14:18:22.771614+0000","last_clean":"2024-09-16T14:18:22.771614+0000","last_became_active":"2024-09-16T14:17:55.492764+0000","last_became_peered":"2024-09-16T14:17:55.492764+0000","last_unstale":"2024-09-16T14:18:22.771614+0000","last_undegraded":"2024-09-16T14:18:22.771614+0000","last_fullsized":"2024-09-16T14:18:22.771614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:17:54.470785+0000","last_clean_scrub_stamp":"2024-09-16T14:17:54.470785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:15.614810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":67,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201302,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32484,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708572,"statfs":{"total":95990841344,"available":95957577728,"internally_reserved":0,"allocated":1404928,"data_stored":928340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60199999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.54000000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495642,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32460,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708596,"statfs":{"total":95990841344,"available":95957602304,"internally_reserved":0,"allocated":1380352,"data_stored":927093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67300000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.45900000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42399999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51100000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31892,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709164,"statfs":{"total":95990841344,"available":95958183936,"internally_reserved":0,"allocated":794624,"data_stored":337401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.622}]},{"osd":1,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67600000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.628},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.27200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.40899999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.44600000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.46600000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709148,"statfs":{"total":95990841344,"available":95958167552,"internally_reserved":0,"allocated":811008,"data_stored":337810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.52500000000000002},"min":{"1min":0.35099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66000000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.499},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.89700000000000002,"15min":0.96899999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":8.2189999999999994,"15min":8.2189999999999994},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.89100000000000001,"15min":0.95599999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66800000000000004,"5min":8.234,"15min":8.234},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55800000000000005,"15min":0.56699999999999995},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66000000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54700000000000004,"15min":0.55200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70399999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.89900000000000002,"15min":0.96799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.61799999999999999,"5min":8.2680000000000007,"15min":8.2680000000000007},"last":0.437}]},{"osd":5,"last update":"Mon Sep 16 14:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.34799999999999998},"min":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.46500000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44700000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.48499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31928,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709128,"statfs":{"total":95990841344,"available":95958147072,"internally_reserved":0,"allocated":831488,"data_stored":338628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.50800000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.435},{"interface":"front","average":{"1min":0.371,"5min":0.40999999999999998,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.29599999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39200000000000002,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48199999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.433,"15min":0.44900000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.379},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.437,"15min":0.44400000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52200000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40100000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.59599999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713256,"statfs":{"total":95990841344,"available":95962374144,"internally_reserved":0,"allocated":798720,"data_stored":337531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57799999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41399999999999998,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.31}]},{"osd":1,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40699999999999997,"15min":0.40400000000000003},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:19:06 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.435,"15min":0.435},"max":{"1min":0.89600000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.623,"15min":0.626},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79900000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77900000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31912,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709144,"statfs":{"total":95990841344,"available":95958163456,"internally_reserved":0,"allocated":815104,"data_stored":337891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34300000000000003,"15min":0.33000000000000002},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.48799999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.37,"15min":0.34799999999999998},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.313}]},{"osd":2,"last update":"Mon Sep 16 14:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.378,"15min":0.36399999999999999},"min":{"1min":0.27700000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.41799999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84999999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.46000000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55500000000000005}]},{"osd":5,"last update":"Mon Sep 16 14:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Sep 16 14:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32480,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708576,"statfs":{"total":95990841344,"available":95957581824,"internally_reserved":0,"allocated":1396736,"data_stored":928205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.34200000000000003,"15min":0.32200000000000001},"min":{"1min":0.26000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.184,"15min":0.184},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.495}]},{"osd":2,"last update":"Mon Sep 16 14:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.26400000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.54000000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.38,"15min":0.372},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.36399999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.62,"15min":0.622},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.79000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:19:15.708 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T14:19:15.708 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T14:19:15.708 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T14:19:15.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T14:19:15.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:15 smithi189 ceph-mon[30490]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T14:19:15.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:15 smithi189 ceph-mon[30490]: Deploying daemon grafana.a on smithi189 2024-09-16T14:19:15.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:15 smithi189 ceph-mon[30490]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:15.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:15 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:15.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:15 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:15.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:15 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:16.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:16 smithi012 ceph-mon[38187]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:19:16.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:16 smithi012 ceph-mon[25528]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:19:16.837 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:16 smithi189 ceph-mon[30490]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T14:19:17.527 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:17 smithi012 ceph-mon[25528]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:17.527 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:17 smithi012 ceph-mon[38187]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:17.688 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T14:19:17.689 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T14:19:17.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:17 smithi189 ceph-mon[30490]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:18.291 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T14:19:18.292 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T14:19:18.292 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T14:19:18.302 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T14:19:18.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-16T14:19:18.512 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:18.633 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:18 smithi012 ceph-mon[38187]: from='client.? 172.21.15.12:0/2704935242' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T14:19:18.633 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:18 smithi012 ceph-mon[25528]: from='client.? 172.21.15.12:0/2704935242' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T14:19:18.837 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:18 smithi189 ceph-mon[30490]: from='client.? 172.21.15.12:0/2704935242' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T14:19:19.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:19 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:19:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:19:19.574 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:19 smithi012 ceph-mon[38187]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:19.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:19 smithi012 ceph-mon[25528]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:19.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:19 smithi189 ceph-mon[30490]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:20.094 INFO:teuthology.orchestra.run.smithi012.stdout:167 167 2024-09-16T14:19:20.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T14:19:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:21.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:21 smithi012 ceph-mon[25528]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:21.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:21 smithi012 ceph-mon[38187]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:21.837 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:21 smithi189 ceph-mon[30490]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:22.799 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-09-16T14:19:22.800 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-09-16T14:19:22.800 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-09-16T14:19:23.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T14:19:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:23.692 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:23 smithi189 ceph-mon[30490]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:23.741 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:23 smithi012 ceph-mon[38187]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:23.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:23 smithi012 ceph-mon[25528]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:23.744 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:19:23 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:19:23.288Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002413729s 2024-09-16T14:19:24.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:19:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:19:24.748 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:24 smithi189 ceph-mon[30490]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:24.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:24 smithi012 ceph-mon[38187]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:24.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:24 smithi012 ceph-mon[25528]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager.a smithi012 *:9093,9094 starting - - - - 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:iscsi.iscsi.a smithi189 *:5000 running (82s) 52s ago 81s 52.4M - 3.8 67ddd33fa72f 4058dc7166f7 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi189 *:8443,8765 running (5m) 52s ago 5m 435M - 18.2.4-845-ge4e227b3 67ddd33fa72f 2e94ab674876 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi012 *:9283,8765 running (6m) 37s ago 6m 456M - 18.2.4-845-ge4e227b3 67ddd33fa72f bd4030ec6967 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (6m) 37s ago 7m 53.5M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f de26539d4f25 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi189 running (5m) 52s ago 5m 48.0M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f 9ed026956892 2024-09-16T14:19:25.408 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi012 running (5m) 37s ago 5m 46.6M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f b0ec5160f6fa 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (4m) 37s ago 4m 54.6M 3761M 18.2.4-845-ge4e227b3 67ddd33fa72f 632f3a696c10 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi012 running (4m) 37s ago 4m 54.8M 3761M 18.2.4-845-ge4e227b3 67ddd33fa72f dfce4edf9186 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi012 running (3m) 37s ago 3m 43.8M 3761M 18.2.4-845-ge4e227b3 67ddd33fa72f 216f39fc85bd 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (3m) 37s ago 3m 50.9M 3761M 18.2.4-845-ge4e227b3 67ddd33fa72f 1cd6a60cff8b 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi189 running (3m) 52s ago 3m 51.4M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 57bf54c314d0 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi189 running (2m) 52s ago 2m 50.5M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 8541f645d081 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi189 running (2m) 52s ago 2m 49.0M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 79ffa81f9451 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi189 running (116s) 52s ago 115s 51.9M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f fa1a1f2f7683 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus.a smithi189 *:9095 running (62s) 52s ago 62s 26.7M - 2.43.0 a07b618ecd1d 9f8db8f19abe 2024-09-16T14:19:25.409 INFO:teuthology.orchestra.run.smithi012.stdout:rgw.foo.a smithi012 *:80 running (96s) 37s ago 95s 73.5M - 18.2.4-845-ge4e227b3 67ddd33fa72f ac3b8513bf6c 2024-09-16T14:19:25.743 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:25 smithi012 ceph-mon[38187]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:25.744 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:25 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:25.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:25 smithi012 ceph-mon[25528]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:25.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:25 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:25.769 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:25 smithi189 ceph-mon[30490]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:25.769 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:25 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:26.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T14:19:26.338 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 systemd[1]: Starting Ceph grafana.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:26.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:26 smithi012 ceph-mon[38187]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:26.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:26 smithi012 ceph-mon[25528]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:26.781 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-mon[30490]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:26.782 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 podman[69879]: 2024-09-16 14:19:26.470998802 +0000 UTC m=+0.105008590 container create 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a, version=8.5, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, release=239.1651231664, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, distribution-scope=public, io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, com.redhat.component=ubi8-container, vendor=Red Hat, Inc.) 2024-09-16T14:19:26.782 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 podman[69879]: 2024-09-16 14:19:26.380744668 +0000 UTC m=+0.014754466 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-16T14:19:26.782 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 podman[69879]: 2024-09-16 14:19:26.68119401 +0000 UTC m=+0.315203809 container init 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, description=Ceph Grafana Container, vendor=Red Hat, Inc., distribution-scope=public, version=8.5, release=239.1651231664, com.redhat.component=ubi8-container, io.openshift.expose-services=, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64) 2024-09-16T14:19:26.782 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 podman[69879]: 2024-09-16 14:19:26.683895727 +0000 UTC m=+0.317905516 container start 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a, com.redhat.component=ubi8-container, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, version=8.5, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container) 2024-09-16T14:19:26.782 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 bash[69879]: 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 2024-09-16T14:19:27.033 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 systemd[1]: Started Ceph grafana.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:27.033 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841363683Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-16T14:19:27.033 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841556302Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-16T14:19:27.033 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841574234Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-16T14:19:27.033 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.84158098Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-16T14:19:27.033 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841586557Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841592172Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841597578Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841603042Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841609058Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841614536Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841619663Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841624865Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841630244Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=settings t=2024-09-16T14:19:26.841635794Z level=info msg="App mode production" 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore t=2024-09-16T14:19:26.841697408Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-16T14:19:27.034 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore t=2024-09-16T14:19:26.841713584Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.842673643Z level=info msg="Starting DB migrations" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.842746648Z level=info msg="Executing migration" id="create migration_log table" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.843264149Z level=info msg="Executing migration" id="create user table" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.843687337Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.844150104Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.844597806Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.845084651Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.845487397Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.846220507Z level=info msg="Executing migration" id="create user table v2" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.846634275Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.847062443Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-16T14:19:27.035 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.847516891Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.847836589Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.848216061Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.848610415Z level=info msg="Executing migration" id="Update user table charset" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.848915764Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.84934461Z level=info msg="Executing migration" id="Add missing user data" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.849692084Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.850156712Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.850525727Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.85095216Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-16T14:19:27.036 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.852102618Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-16T14:19:27.037 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.852547182Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.85301001Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.853412264Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.853816408Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.854235519Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.854495902Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.854869706Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.855259425Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.855617689Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.856000144Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.856628224Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.857060551Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.857468105Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.85782576Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-16T14:19:27.038 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.858327788Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.85892285Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.859355412Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.859906542Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.860240669Z level=info msg="Executing migration" id="create star table" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.860569427Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.861039497Z level=info msg="Executing migration" id="create org table v1" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.861436582Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.863778978Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.864179779Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.864577905Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.865004197Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.865406557Z level=info msg="Executing migration" id="Update org table charset" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.865673554Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.865979447Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-16T14:19:27.039 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.866295446Z level=info msg="Executing migration" id="create dashboard table" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.866660359Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.867082656Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.867501212Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.868359637Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.869149769Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.869542429Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.870425132Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.871288421Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.872134322Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.872566686Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.873568017Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.874302102Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.875285862Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.875931434Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-16T14:19:27.040 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.876923855Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.877608171Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.87863009Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.879210164Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.880068268Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.880503977Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.880886642Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.881211528Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.881772883Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.882302936Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.88521004Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.885963858Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.888087544Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.890856313Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.891247009Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-16T14:19:27.041 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.894181004Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.894623254Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.897010581Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.899640914Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.900070964Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.902900572Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.905723867Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.908571516Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.911306195Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.914057063Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.916797561Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.919730805Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.922484934Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.925157073Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.927984334Z level=info msg="Executing migration" id="create data_source table" 2024-09-16T14:19:27.042 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.93041355Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.933243488Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.935923535Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.938714716Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.941324002Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.944114768Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.946801537Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.949504833Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.952185651Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.954887236Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.957630773Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.960200537Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.962928986Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-16T14:19:27.043 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.965668455Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.96839791Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.969251648Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.969992423Z level=info msg="Executing migration" id="Add uid column" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.970806489Z level=info msg="Executing migration" id="Update uid value" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.971680409Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.972466896Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.973425726Z level=info msg="Executing migration" id="create api_key table" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.974140677Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.974967728Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.975866342Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.976663598Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.977625022Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-16T14:19:27.044 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.978462741Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.97920467Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.980328665Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.981192198Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.982019725Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.982866295Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.983303562Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.984114685Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.9845047Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.985047971Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.985854905Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.986432342Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.986711502Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.98730058Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.98785952Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-16T14:19:27.045 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.988309629Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.988685177Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.989122926Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.989566419Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.990011721Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.990846503Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.991689424Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.992535918Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.993131043Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.995968274Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:26.998940101Z level=info msg="Executing migration" id="create quota table v1" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.001706581Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-16T14:19:27.046 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.002179584Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.004947927Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.005353299Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.007701881Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.010590912Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.01095742Z level=info msg="Executing migration" id="create session table" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.013991552Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.016898798Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.019656448Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.020074265Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.022984771Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.025678103Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.028437556Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.028867063Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-16T14:19:27.047 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.031846796Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-16T14:19:27.048 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.034495322Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.037313157Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.039850792Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.04262322Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.045343331Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.04791661Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.048218624Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-16T14:19:27.285 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.05099278Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.053744642Z level=info msg="Executing migration" id="create alert table v1" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.05649683Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.057006221Z level=info msg="Executing migration" id="add index alert state" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.059790506Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.062468441Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.065255544Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.065796697Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.068703966Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.071166795Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.072135162Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.072698975Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.073468136Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.07438114Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.075192414Z level=info msg="Executing migration" id="Add column is_default" 2024-09-16T14:19:27.286 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.075858742Z level=info msg="Executing migration" id="Add column frequency" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.076639853Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.077454268Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.07838576Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.079001221Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.079280683Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.079543443Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.079931114Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.080345803Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.080690341Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.081080505Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.081486681Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.082121133Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.082783238Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.083093957Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.083502945Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-16T14:19:27.287 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.083921948Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.084578815Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.084848759Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.085262752Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.0856575Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.0859624Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.086380233Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.086783656Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.087196805Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.087610374Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.088050296Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.088494357Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.088749743Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.089464545Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-16T14:19:27.288 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.090311074Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-16T14:19:27.289 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.091245023Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-16T14:19:27.289 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.091650192Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-16T14:19:27.289 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.09207073Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-16T14:19:27.289 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.092857982Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-16T14:19:27.289 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.094502831Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-16T14:19:27.290 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.095304797Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-16T14:19:27.290 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.095770395Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-16T14:19:27.290 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.096623494Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-16T14:19:27.290 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.09749918Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.098277101Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.099004182Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.099791863Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.100621416Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.101540538Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.101982981Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.102806825Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.103656683Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.104553143Z level=info msg="Executing migration" id="Move region to single row" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.104944675Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.105950376Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.106646497Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.107628408Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-16T14:19:27.291 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.108166959Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.109086971Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.109696209Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.110037936Z level=info msg="Executing migration" id="create test_data table" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.110435164Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.110870887Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.11175346Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.112582539Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.113393244Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.113735262Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.114598677Z level=info msg="Executing migration" id="create team table" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.117529192Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.120373891Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.123092357Z level=info msg="Executing migration" id="create team member table" 2024-09-16T14:19:27.292 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.125840211Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.128484699Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.131405578Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.131931349Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.134601781Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.137398768Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.140074377Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.140511723Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.143353056Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.145955446Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.146381964Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.146852895Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.149558204Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-16T14:19:27.293 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.152071475Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.154829299Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.157580345Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.160129581Z level=info msg="Executing migration" id="create tag table" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.162896458Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.165563546Z level=info msg="Executing migration" id="create login attempt table" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.168234427Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.170995876Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.173653897Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.176333415Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.178885212Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.181555854Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.184215958Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.18693668Z level=info msg="Executing migration" id="create user auth table" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.189636108Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-16T14:19:27.294 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.192131314Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.194868377Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.197474875Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.199959947Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.202821505Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.20378295Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.20457971Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.205395096Z level=info msg="Executing migration" id="create server_lock table" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.206180493Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.207031726Z level=info msg="Executing migration" id="create user auth token table" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.207828622Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.208602441Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-16T14:19:27.295 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.209396652Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.210180151Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.211015126Z level=info msg="Executing migration" id="create cache_data table" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.211873836Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.21264066Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.213579398Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.214303802Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.21497653Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.215297531Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.215804314Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-16T14:19:27.296 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.216499553Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.216959058Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.217270143Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.217677517Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.21814653Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.218583745Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.219033565Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.2198471Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.220287692Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.220566557Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.22100922Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.221457866Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.221891058Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.222188652Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-16T14:19:27.297 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.222575087Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.223006067Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.223437032Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.223880233Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.22470138Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.225162349Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.225588287Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.229266929Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.23281229Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.233290818Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.234115989Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.235179791Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.235632234Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.236507206Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-16T14:19:27.298 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.237572141Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.238242719Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.239207557Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.240205309Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.241035351Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.241846615Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.242722082Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.243601656Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.244487481Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.24535602Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-16T14:19:27.299 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.246211988Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.24712021Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.24799329Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.248858319Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.249864606Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.250573403Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.25144368Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.252288288Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.253237354Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.25412951Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.254993031Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.255855987Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.25670645Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.257581112Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-16T14:19:27.300 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.258555047Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.261252107Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.264019955Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.266856343Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.269727908Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.272540118Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.275328576Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.278162745Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.28093366Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.28364035Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-16T14:19:27.301 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.286464926Z level=info msg="Executing migration" id="create alert_image table" 2024-09-16T14:19:27.552 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[38187]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:27.552 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.552 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.552 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.552 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[25528]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:27.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:27 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-mon[30490]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:27.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.561 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:27.561 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.289388173Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.29232519Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.295096258Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.297920071Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.300810503Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.30088905Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.303489197Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.306299369Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.30897075Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.309088016Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.309197864Z level=info msg="alerts found to migrate" alerts=0 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.311716377Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.314456788Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-16T14:19:27.562 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.317188243Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-16T14:19:27.563 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.320056201Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-16T14:19:27.563 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.322813138Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-16T14:19:27.563 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.325449042Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-16T14:19:27.563 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.328185534Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-16T14:19:27.563 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.331011475Z level=info msg="Executing migration" id="create data_keys table" 2024-09-16T14:19:27.563 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.33362541Z level=info msg="Executing migration" id="create secrets table" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.336331565Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.340872072Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.343609603Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.346340053Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.350872271Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.355582181Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.358262176Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.360974839Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.361802271Z level=info msg="Executing migration" id="create permission table" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.362851037Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.363463531Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.364468152Z level=info msg="Executing migration" id="create role table" 2024-09-16T14:19:27.564 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.365136684Z level=info msg="Executing migration" id="add column display_name" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.366128453Z level=info msg="Executing migration" id="add column group_name" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.367122616Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.367979462Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.368789936Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.369596468Z level=info msg="Executing migration" id="create team role table" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.370442265Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.371417562Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.372158109Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.372938458Z level=info msg="Executing migration" id="create user role table" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.373722403Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.374652652Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.375358723Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-16T14:19:27.565 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.376330127Z level=info msg="Executing migration" id="create builtin role table" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.37707579Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.377954491Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.378809506Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.379802021Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.380649414Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.381661371Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.382417233Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.383296481Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.38382624Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.384302184Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.385326997Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.386179817Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.386942432Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.387890045Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-16T14:19:27.566 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.388744639Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.389483198Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.390331644Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.390672358Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.393265351Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.39592579Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.39886985Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.401729964Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.404433353Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.407154682Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.409986317Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.412787898Z level=info msg="Executing migration" id="add correlation config column" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.415701991Z level=info msg="Executing migration" id="create entity_events table" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.41858432Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-16T14:19:27.567 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.42164997Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T14:19:27.568 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.421955452Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T14:19:27.568 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.424411046Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T14:19:27.568 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.424514834Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T14:19:27.568 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.427245992Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-16T14:19:27.568 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.429878603Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-16T14:19:27.568 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.432580175Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.435575853Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.438205954Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.441089761Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.443553818Z level=info msg="Executing migration" id="Drop public config table" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.446143821Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.448943131Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.451726876Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.454450636Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.457012257Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.460049343Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.462950177Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.465362614Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.468085188Z level=info msg="Executing migration" id="add share column" 2024-09-16T14:19:27.569 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.469283741Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.469811826Z level=info msg="Executing migration" id="create file table" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.470289304Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.470746686Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.471254975Z level=info msg="Executing migration" id="create file_meta table" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.472081704Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.472566749Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.473496794Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.474207644Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.475105527Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.47574468Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.476816739Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.47764091Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-16T14:19:27.570 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.478491743Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.47895977Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.479740069Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.480573055Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.481649553Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.482739944Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.483590273Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.493709925Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.494552616Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.495062489Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.498606473Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.499516441Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.500288612Z level=info msg="Executing migration" id="create folder table" 2024-09-16T14:19:27.571 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.500734015Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-16T14:19:27.572 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.501640044Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-16T14:19:27.572 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.502608225Z level=info msg="Executing migration" id="Update folder title length" 2024-09-16T14:19:27.572 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.503437839Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-16T14:19:27.572 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=migrator t=2024-09-16T14:19:27.504133007Z level=info msg="migrations completed" performed=485 skipped=0 duration=661.404167ms 2024-09-16T14:19:27.572 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore t=2024-09-16T14:19:27.504568606Z level=info msg="Created default organization" 2024-09-16T14:19:27.572 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=plugin.loader t=2024-09-16T14:19:27.560588701Z level=info msg="Plugin registered" pluginID=input 2024-09-16T14:19:27.838 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=plugin.loader t=2024-09-16T14:19:27.58962182Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-16T14:19:27.838 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=plugin.loader t=2024-09-16T14:19:27.58965269Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-16T14:19:27.838 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=secrets t=2024-09-16T14:19:27.589689165Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-16T14:19:27.838 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=query_data t=2024-09-16T14:19:27.591288235Z level=info msg="Query Service initialization" 2024-09-16T14:19:27.838 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=live.push_http t=2024-09-16T14:19:27.593608614Z level=info msg="Live Push Gateway initialization" 2024-09-16T14:19:28.277 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:28 smithi012 systemd[1]: Starting Ceph node-exporter.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:28.327 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:alertmanager ?:9093,9094 0/1 - 79s smithi012=a;count:1 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:grafana ?:3000 0/1 - 75s smithi189=a;count:1 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:iscsi.datapool ?:5000 1/1 55s ago 90s smithi189=iscsi.a;count:1 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 2/2 55s ago 5m smithi012=y;smithi189=x;count:2 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 55s ago 5m smithi012:172.21.15.12=a;smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c;smithi189:172.21.15.189=b;count:3 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:node-exporter ?:9100 0/2 - 82s smithi012=a;smithi189=b;count:2 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:osd 8 55s ago - 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:prometheus ?:9095 1/1 55s ago 86s smithi189=a;count:1 2024-09-16T14:19:28.328 INFO:teuthology.orchestra.run.smithi012.stdout:rgw.foo.a ?:80 1/1 40s ago 97s smithi012=foo.a;count:1 2024-09-16T14:19:28.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-mon[30490]: Deploying daemon node-exporter.a on smithi012 2024-09-16T14:19:28.588 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=infra.usagestats.collector t=2024-09-16T14:19:28.327561324Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-16T14:19:28.588 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=provisioning.datasources t=2024-09-16T14:19:28.382180171Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-16T14:19:28.588 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=provisioning.datasources t=2024-09-16T14:19:28.41389723Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-16T14:19:28.588 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=provisioning.alerting t=2024-09-16T14:19:28.435211293Z level=info msg="starting to provision alerting" 2024-09-16T14:19:28.588 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=provisioning.alerting t=2024-09-16T14:19:28.435249641Z level=info msg="finished to provision alerting" 2024-09-16T14:19:28.588 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=ngalert.state.manager t=2024-09-16T14:19:28.435378132Z level=info msg="Warming state cache for startup" 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=grafanaStorageLogger t=2024-09-16T14:19:28.435406815Z level=info msg="storage starting" 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=http.server t=2024-09-16T14:19:28.43726968Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=ngalert.state.manager t=2024-09-16T14:19:28.445581762Z level=info msg="State cache has been initialized" states=0 duration=10.202471ms 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=ticker t=2024-09-16T14:19:28.445646576Z level=info msg=starting first_tick=2024-09-16T14:19:30Z 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=ngalert.multiorg.alertmanager t=2024-09-16T14:19:28.445660178Z level=info msg="starting MultiOrg Alertmanager" 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore.transactions t=2024-09-16T14:19:28.446657999Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore.transactions t=2024-09-16T14:19:28.487393749Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore.transactions t=2024-09-16T14:19:28.517598459Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T14:19:28.589 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:19:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a[69894]: logger=sqlstore.transactions t=2024-09-16T14:19:28.528209434Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T14:19:28.612 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:28 smithi012 ceph-mon[38187]: Deploying daemon node-exporter.a on smithi012 2024-09-16T14:19:28.613 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:28 smithi012 bash[79784]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T14:19:28.613 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:28 smithi012 ceph-mon[25528]: Deploying daemon node-exporter.a on smithi012 2024-09-16T14:19:28.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T14:19:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:29.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:29 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:19:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:19:29.542 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:29 smithi012 ceph-mon[38187]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:29.543 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:29 smithi012 ceph-mon[38187]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:29.543 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:29 smithi012 ceph-mon[25528]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:29.543 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:29 smithi012 ceph-mon[25528]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:29 smithi189 ceph-mon[30490]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:29.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:29 smithi189 ceph-mon[30490]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:31.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:30 smithi012 ceph-mon[38187]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:31.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:30 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:31.017 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:30 smithi012 ceph-mon[38187]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:31.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:30 smithi012 ceph-mon[25528]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:31.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:30 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:31.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:30 smithi012 ceph-mon[25528]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:31.018 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:30 smithi012 bash[79784]: Getting image source signatures 2024-09-16T14:19:31.018 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:30 smithi012 bash[79784]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T14:19:31.018 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:30 smithi012 bash[79784]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T14:19:31.018 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:30 smithi012 bash[79784]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T14:19:31.018 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-09-16T14:19:31.018 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-09-16T14:19:31.018 INFO:teuthology.orchestra.run.smithi012.stdout:smithi189 172.21.15.189 2024-09-16T14:19:31.018 INFO:teuthology.orchestra.run.smithi012.stdout:2 hosts in cluster 2024-09-16T14:19:31.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:30 smithi189 ceph-mon[30490]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:31.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:30 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:31.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:30 smithi189 ceph-mon[30490]: from='mgr.14589 172.21.15.12:0/2708417044' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T14:19:32.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:31 smithi012 ceph-mon[25528]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:32.027 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:31 smithi012 ceph-mon[38187]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:32.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:31 smithi189 ceph-mon[30490]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:32.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T14:19:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:32.841 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:32 smithi012 ceph-mon[38187]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:32.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:32 smithi012 ceph-mon[25528]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:32.841 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:32 smithi012 bash[79784]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T14:19:32.842 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:32 smithi012 bash[79784]: Writing manifest to image destination 2024-09-16T14:19:33.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:32 smithi189 ceph-mon[30490]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:33.448 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:33 smithi012 podman[79784]: 2024-09-16 14:19:33.138254083 +0000 UTC m=+4.901479315 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T14:19:34.329 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 podman[79784]: 2024-09-16 14:19:34.067137157 +0000 UTC m=+5.830362390 container create dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T14:19:34.337 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:19:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:19:34.777 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 podman[79784]: 2024-09-16 14:19:34.328325165 +0000 UTC m=+6.091550396 container init dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T14:19:34.777 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 podman[79784]: 2024-09-16 14:19:34.332116948 +0000 UTC m=+6.095342180 container start dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.337Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.337Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.337Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.337Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T14:19:34.778 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T14:19:34.779 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T14:19:34.780 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T14:19:34.781 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.338Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.339Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a[80135]: ts=2024-09-16T14:19:34.339Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 bash[79784]: dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 2024-09-16T14:19:34.782 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:19:34 smithi012 systemd[1]: Started Ceph node-exporter.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:35.609 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T14:19:35.609 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T14:19:35.609 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme1n1 ssd Linux_b0914b709fa4950f04ca 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme2n1 ssd Linux_9333e8110a13490cf305 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme3n1 ssd Linux_2c380defbffe8e1f4fb0 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme4n1 ssd Linux_84776b20176c1979cb3b 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 117s ago Has a FileSystem, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi189 /dev/nvme1n1 ssd Linux_abba6dffe417aee506d0 89.4G No 117s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi189 /dev/nvme2n1 ssd Linux_94773ca22734a2a60893 89.4G No 117s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi189 /dev/nvme3n1 ssd Linux_e9a18bb23920b2aaeff1 89.4G No 117s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.610 INFO:teuthology.orchestra.run.smithi012.stdout:smithi189 /dev/nvme4n1 ssd Linux_b535c61445c2800adb5b 89.4G No 117s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T14:19:35.738 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:35 smithi189 ceph-mon[30490]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:35.739 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:35 smithi189 ceph-mon[30490]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:35.739 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:35 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.739 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:35 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.739 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:35 smithi189 ceph-mon[30490]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[25528]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[25528]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[25528]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[38187]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:35.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[38187]: from='client.14538 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T14:19:35.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:35.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:35 smithi012 ceph-mon[38187]: from='mgr.14589 ' entity='mgr.y' 2024-09-16T14:19:36.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-16T14:19:36.542 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:36.663 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:36 smithi012 ceph-mon[25528]: Deploying daemon node-exporter.b on smithi189 2024-09-16T14:19:36.663 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:36 smithi012 ceph-mon[38187]: Deploying daemon node-exporter.b on smithi189 2024-09-16T14:19:36.814 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:36 smithi189 ceph-mon[30490]: Deploying daemon node-exporter.b on smithi189 2024-09-16T14:19:36.814 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:36 smithi189 systemd[1]: Starting Ceph node-exporter.b for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:37.088 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:36 smithi189 bash[70150]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T14:19:37.481 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:37 smithi189 bash[70150]: Getting image source signatures 2024-09-16T14:19:37.481 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:37 smithi189 bash[70150]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T14:19:37.481 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:37 smithi189 bash[70150]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T14:19:37.481 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:37 smithi189 bash[70150]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T14:19:37.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:37 smithi012 ceph-mon[25528]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:37.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:37 smithi012 ceph-mon[25528]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:37.777 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:37 smithi012 ceph-mon[38187]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:37.778 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:37 smithi012 ceph-mon[38187]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:37.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:37 smithi189 ceph-mon[30490]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T14:19:37.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:37 smithi189 ceph-mon[30490]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: alertmanager 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: alertmanager 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: count: 1 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi012=a 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:18:09.294370Z' 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: ports: 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: - 9093 2024-09-16T14:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout: - 9094 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: running: 0 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: size: 1 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:events: 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:- 2024-09-16T14:19:13.449444Z service:alertmanager [INFO] "service was created" 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: grafana 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: grafana 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: count: 1 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi189=a 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:spec: 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: anonymous_access: true 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout: protocol: https 2024-09-16T14:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:18:12.350172Z' 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: ports: 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: - 3000 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: running: 0 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: size: 1 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:events: 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:- 2024-09-16T14:19:26.869003Z service:grafana [INFO] "service was created" 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: iscsi 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:service_id: datapool 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: iscsi.datapool 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: count: 1 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi189=iscsi.a 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout:spec: 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: api_password: admin 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: api_port: 5000 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: api_user: admin 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: pool: datapool 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: trusted_ip_list: 172.21.15.189 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:17:58.150058Z' 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: last_refresh: '2024-09-16T14:18:32.489270Z' 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: ports: 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: - 5000 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: running: 1 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout: size: 1 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.324 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: mgr 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: mgr 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: count: 2 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi012=y 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi189=x 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:14:17.550315Z' 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: last_refresh: '2024-09-16T14:18:32.488953Z' 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: running: 2 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: size: 2 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: mon 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: mon 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.325 INFO:teuthology.orchestra.run.smithi012.stdout: count: 3 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi012:172.21.15.12=a 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi012:[v2:172.21.15.12:3301,v1:172.21.15.12:6790]=c 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi189:172.21.15.189=b 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:13:50.669608Z' 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: last_refresh: '2024-09-16T14:18:32.488854Z' 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: running: 3 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: size: 3 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: node-exporter 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: node-exporter 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: count: 2 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.326 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi012=a 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi189=b 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:18:05.566368Z' 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: ports: 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: - 9100 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: running: 0 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: size: 2 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:events: 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:- 2024-09-16T14:19:34.486282Z service:node-exporter [INFO] "service was created" 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: osd 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: osd 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:unmanaged: true 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout:spec: 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: filter_logic: AND 2024-09-16T14:19:38.327 INFO:teuthology.orchestra.run.smithi012.stdout: objectstore: bluestore 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: container_image_id: 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: last_refresh: '2024-09-16T14:18:32.489022Z' 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: running: 8 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: size: 8 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: prometheus 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: prometheus 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: count: 1 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi189=a 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.328 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:18:02.244562Z' 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: last_refresh: '2024-09-16T14:18:32.489331Z' 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: ports: 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: - 9095 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: running: 1 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: size: 1 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:--- 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:service_type: rgw 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:service_id: foo.a 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:service_name: rgw.foo.a 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:placement: 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: count: 1 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: hosts: 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: - smithi012=foo.a 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout:status: 2024-09-16T14:19:38.329 INFO:teuthology.orchestra.run.smithi012.stdout: created: '2024-09-16T14:17:50.411284Z' 2024-09-16T14:19:38.330 INFO:teuthology.orchestra.run.smithi012.stdout: last_refresh: '2024-09-16T14:18:48.086481Z' 2024-09-16T14:19:38.330 INFO:teuthology.orchestra.run.smithi012.stdout: ports: 2024-09-16T14:19:38.330 INFO:teuthology.orchestra.run.smithi012.stdout: - 80 2024-09-16T14:19:38.330 INFO:teuthology.orchestra.run.smithi012.stdout: running: 1 2024-09-16T14:19:38.330 INFO:teuthology.orchestra.run.smithi012.stdout: size: 1 2024-09-16T14:19:38.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-16T14:19:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T14:19:39.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:39 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: ::ffff:172.21.15.189 - - [16/Sep/2024:14:19:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T14:19:39.633 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:39 smithi012 ceph-mon[25528]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:39.633 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:39 smithi012 ceph-mon[25528]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T14:19:39.633 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:39 smithi012 ceph-mon[38187]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:39.633 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:39 smithi012 ceph-mon[38187]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T14:19:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:39 smithi189 ceph-mon[30490]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:39.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:39 smithi189 ceph-mon[30490]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T14:19:40.701 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:40 smithi012 ceph-mon[25528]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:40.702 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:40 smithi012 ceph-mon[38187]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:40.838 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:40 smithi189 ceph-mon[30490]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T14:19:41.094 INFO:teuthology.orchestra.run.smithi012.stdout:osd 8 68s ago - 2024-09-16T14:19:41.753 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:41 smithi012 ceph-mon[38187]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:41.753 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:41 smithi012 ceph-mon[25528]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:41.775 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T14:19:41.911 INFO:tasks.cephadm:Teardown begin 2024-09-16T14:19:41.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:19:41.913 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:41 smithi189 ceph-mon[30490]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T14:19:41.913 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:41 smithi189 bash[70150]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T14:19:41.913 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:41 smithi189 bash[70150]: Writing manifest to image destination 2024-09-16T14:19:41.943 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:19:41.973 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T14:19:41.974 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T14:19:42.000 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T14:19:42.030 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T14:19:42.031 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T14:19:42.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a 2024-09-16T14:19:42.338 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 podman[70150]: 2024-09-16 14:19:41.963789267 +0000 UTC m=+5.243844637 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T14:19:42.338 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 podman[70150]: 2024-09-16 14:19:42.07917655 +0000 UTC m=+5.359231957 container create 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T14:19:42.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:42 smithi012 systemd[1]: Stopping Ceph mon.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:42.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a[25524]: 2024-09-16T14:19:42.338+0000 7f2d8fa90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:19:42.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:42 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a[25524]: 2024-09-16T14:19:42.338+0000 7f2d8fa90640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T14:19:42.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:42 smithi012 podman[80615]: 2024-09-16 14:19:42.553838797 +0000 UTC m=+0.324083440 container died de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T14:19:42.838 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 podman[70150]: 2024-09-16 14:19:42.36470812 +0000 UTC m=+5.644763478 container init 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T14:19:42.838 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 podman[70150]: 2024-09-16 14:19:42.367686939 +0000 UTC m=+5.647742301 container start 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T14:19:42.838 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.371Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.371Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T14:19:42.839 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T14:19:42.840 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T14:19:42.841 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T14:19:42.842 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T14:19:42.843 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T14:19:42.843 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.372Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T14:19:42.843 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.373Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T14:19:42.843 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b[70258]: ts=2024-09-16T14:19:42.373Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T14:19:42.843 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 bash[70150]: 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 2024-09-16T14:19:42.843 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:19:42 smithi189 systemd[1]: Started Ceph node-exporter.b for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:43.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:43 smithi012 podman[80615]: 2024-09-16 14:19:43.005596228 +0000 UTC m=+0.775840870 container cleanup de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-16T14:19:43.277 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:43 smithi012 bash[80615]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a 2024-09-16T14:19:43.608 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T14:19:43.631 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:43 smithi012 podman[80627]: 2024-09-16 14:19:43.365637719 +0000 UTC m=+0.810095571 container remove de26539d4f25a608c40b9c6c900f84333f84e60d29e907ad040e40bdc8922bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b) 2024-09-16T14:19:43.631 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:43 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T14:19:43.632 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:43 smithi012 systemd[1]: Stopped Ceph mon.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:43.632 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 16 14:19:43 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.a.service: Consumed 11.745s CPU time. 2024-09-16T14:19:43.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:19:43.655 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T14:19:43.655 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-16T14:19:43.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.c 2024-09-16T14:19:43.978 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:43 smithi012 systemd[1]: Stopping Ceph mon.c for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:43 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:43] ENGINE Bus STOPPING 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:44] ENGINE Bus STOPPED 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:44] ENGINE Bus STARTING 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:44] ENGINE Serving on http://:::9283 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:44 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: [16/Sep/2024:14:19:44] ENGINE Bus STARTED 2024-09-16T14:19:44.277 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:43 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c[38183]: 2024-09-16T14:19:43.976+0000 7f37eedd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:19:44.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:43 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c[38183]: 2024-09-16T14:19:43.976+0000 7f37eedd7640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T14:19:44.278 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:43 smithi012 podman[80738]: 2024-09-16 14:19:43.997819977 +0000 UTC m=+0.114145561 container died b0ec5160f6fa8ffd6cd55ddce72465cfd45d24c0932fa6facccb7c8e36f8f798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T14:19:44.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:19:43 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug there is no tcmu-runner data available 2024-09-16T14:19:44.738 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:44 smithi012 podman[80738]: 2024-09-16 14:19:44.452113728 +0000 UTC m=+0.568439294 container cleanup b0ec5160f6fa8ffd6cd55ddce72465cfd45d24c0932fa6facccb7c8e36f8f798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T14:19:44.738 INFO:journalctl@ceph.mon.c.smithi012.stdout:Sep 16 14:19:44 smithi012 bash[80738]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-c 2024-09-16T14:19:44.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T14:19:44.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:19:44.921 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-16T14:19:44.921 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-16T14:19:44.921 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.b 2024-09-16T14:19:45.237 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:44 smithi189 systemd[1]: Stopping Ceph mon.b for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:45.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:45 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-b[30485]: 2024-09-16T14:19:45.234+0000 7fddb8f90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:19:45.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:45 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-b[30485]: 2024-09-16T14:19:45.234+0000 7fddb8f90640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T14:19:45.588 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:45 smithi189 podman[70320]: 2024-09-16 14:19:45.430747917 +0000 UTC m=+0.319016800 container died 9ed02695689210ccea32fbbd7160d95a7ef54dc92bbd42976f09fbae95dd6a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T14:19:46.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:45 smithi189 podman[70320]: 2024-09-16 14:19:45.803649243 +0000 UTC m=+0.691918128 container cleanup 9ed02695689210ccea32fbbd7160d95a7ef54dc92bbd42976f09fbae95dd6a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T14:19:46.088 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 16 14:19:45 smithi189 bash[70320]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mon-b 2024-09-16T14:19:46.383 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T14:19:46.453 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:19:46.453 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-16T14:19:46.453 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T14:19:46.453 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y 2024-09-16T14:19:46.741 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:46 smithi012 systemd[1]: Stopping Ceph mgr.y for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:47.017 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:19:46.739+0000 7fe3aecbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:19:47.017 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:46 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y[25754]: 2024-09-16T14:19:46.739+0000 7fe3aecbd640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T14:19:47.017 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:46 smithi012 podman[80870]: 2024-09-16 14:19:46.771391541 +0000 UTC m=+0.124565469 container died bd4030ec6967ecdda51291404d2113f2455822ea561a5f76a9569e29cf457625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, ceph=True) 2024-09-16T14:19:47.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:47 smithi012 podman[80870]: 2024-09-16 14:19:47.198701738 +0000 UTC m=+0.551875665 container cleanup bd4030ec6967ecdda51291404d2113f2455822ea561a5f76a9569e29cf457625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:19:47.277 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:47 smithi012 bash[80870]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y 2024-09-16T14:19:47.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T14:19:47.648 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:47 smithi012 podman[80883]: 2024-09-16 14:19:47.375114949 +0000 UTC m=+0.602927629 container remove bd4030ec6967ecdda51291404d2113f2455822ea561a5f76a9569e29cf457625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T14:19:47.648 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:47 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service: Deactivated successfully. 2024-09-16T14:19:47.648 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:47 smithi012 systemd[1]: Stopped Ceph mgr.y for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:47.648 INFO:journalctl@ceph.mgr.y.smithi012.stdout:Sep 16 14:19:47 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.y.service: Consumed 40.478s CPU time. 2024-09-16T14:19:47.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:19:47.662 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T14:19:47.662 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T14:19:47.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.x 2024-09-16T14:19:48.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:47 smithi189 systemd[1]: Stopping Ceph mgr.x for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:48.277 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 podman[70442]: 2024-09-16 14:19:48.012552225 +0000 UTC m=+0.152634260 container died 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-e4e227b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:19:48.535 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 podman[70442]: 2024-09-16 14:19:48.358304792 +0000 UTC m=+0.498386817 container cleanup 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T14:19:48.535 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 bash[70442]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x 2024-09-16T14:19:48.781 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T14:19:48.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 podman[70455]: 2024-09-16 14:19:48.534509411 +0000 UTC m=+0.513034923 container remove 2e94ab674876fa42b587282584772ddf5de196c9c4fad89b1ad60009dacf28d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T14:19:48.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-16T14:19:48.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-16T14:19:48.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 systemd[1]: Stopped Ceph mgr.x for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:19:48.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 16 14:19:48 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@mgr.x.service: Consumed 10.790s CPU time. 2024-09-16T14:19:48.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:19:48.817 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T14:19:48.817 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T14:19:48.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.0 2024-09-16T14:19:49.137 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:48 smithi012 systemd[1]: Stopping Ceph osd.0 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:49.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:49 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0[42022]: 2024-09-16T14:19:49.136+0000 7fa7215c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:19:49.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:49 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0[42022]: 2024-09-16T14:19:49.136+0000 7fa7215c5640 -1 osd.0 72 *** Got signal Terminated *** 2024-09-16T14:19:49.527 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:49 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0[42022]: 2024-09-16T14:19:49.136+0000 7fa7215c5640 -1 osd.0 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:19:54.880 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:54 smithi012 podman[80992]: 2024-09-16 14:19:54.613447387 +0000 UTC m=+5.599755068 container died 632f3a696c10ef40c6beeabf2f5297a31de445e6b8043281f7732add415d4bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T14:19:55.277 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 podman[80992]: 2024-09-16 14:19:55.044299077 +0000 UTC m=+6.030606756 container cleanup 632f3a696c10ef40c6beeabf2f5297a31de445e6b8043281f7732add415d4bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T14:19:55.277 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 bash[80992]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0 2024-09-16T14:19:55.722 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 podman[81008]: 2024-09-16 14:19:55.31270159 +0000 UTC m=+0.694796625 container remove 632f3a696c10ef40c6beeabf2f5297a31de445e6b8043281f7732add415d4bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T14:19:56.027 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 podman[81095]: 2024-09-16 14:19:55.721708701 +0000 UTC m=+0.110711911 container create 6e4b42cb00650b22b728dafd64d1d660a50bf428c2f4b05a68452f90578c5fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T14:19:56.027 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 podman[81095]: 2024-09-16 14:19:55.627651049 +0000 UTC m=+0.016654229 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:19:56.027 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 podman[81095]: 2024-09-16 14:19:55.960658453 +0000 UTC m=+0.349661621 container init 6e4b42cb00650b22b728dafd64d1d660a50bf428c2f4b05a68452f90578c5fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:19:56.027 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:55 smithi012 podman[81095]: 2024-09-16 14:19:55.964243754 +0000 UTC m=+0.353246921 container start 6e4b42cb00650b22b728dafd64d1d660a50bf428c2f4b05a68452f90578c5fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=reef-e4e227b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T14:19:56.432 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:56 smithi012 podman[81095]: 2024-09-16 14:19:56.052643076 +0000 UTC m=+0.441646259 container attach 6e4b42cb00650b22b728dafd64d1d660a50bf428c2f4b05a68452f90578c5fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T14:19:56.432 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 16 14:19:56 smithi012 podman[81095]: 2024-09-16 14:19:56.164861746 +0000 UTC m=+0.553864931 container died 6e4b42cb00650b22b728dafd64d1d660a50bf428c2f4b05a68452f90578c5fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T14:19:56.801 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T14:19:56.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:19:56.842 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T14:19:56.842 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T14:19:56.842 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.1 2024-09-16T14:19:57.177 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:19:56 smithi012 systemd[1]: Stopping Ceph osd.1 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:19:57.527 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:19:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1[49815]: 2024-09-16T14:19:57.175+0000 7f7774ad9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:19:57.527 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:19:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1[49815]: 2024-09-16T14:19:57.175+0000 7f7774ad9640 -1 osd.1 72 *** Got signal Terminated *** 2024-09-16T14:19:57.527 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:19:57 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1[49815]: 2024-09-16T14:19:57.175+0000 7f7774ad9640 -1 osd.1 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:02.978 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:02 smithi012 podman[81209]: 2024-09-16 14:20:02.684195986 +0000 UTC m=+5.592433498 container died dfce4edf918632a9b7f8a0e4388eb4e145ea6c9519383f00f0afcc9dafe7891f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-e4e227b, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T14:20:03.277 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 podman[81209]: 2024-09-16 14:20:03.113729666 +0000 UTC m=+6.021967219 container cleanup dfce4edf918632a9b7f8a0e4388eb4e145ea6c9519383f00f0afcc9dafe7891f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T14:20:03.277 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 bash[81209]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1 2024-09-16T14:20:03.746 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 podman[81228]: 2024-09-16 14:20:03.323321814 +0000 UTC m=+0.639233918 container remove dfce4edf918632a9b7f8a0e4388eb4e145ea6c9519383f00f0afcc9dafe7891f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T14:20:03.998 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 podman[81318]: 2024-09-16 14:20:03.648536848 +0000 UTC m=+0.016511468 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:03.999 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 podman[81318]: 2024-09-16 14:20:03.758098523 +0000 UTC m=+0.126073152 container create 84e5b36bf2748fbb2b02457811c420c3982b92dd26d5661d886d40497d046692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T14:20:03.999 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 podman[81318]: 2024-09-16 14:20:03.994348021 +0000 UTC m=+0.362322641 container init 84e5b36bf2748fbb2b02457811c420c3982b92dd26d5661d886d40497d046692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_BRANCH=HEAD) 2024-09-16T14:20:03.999 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:03 smithi012 podman[81318]: 2024-09-16 14:20:03.997958871 +0000 UTC m=+0.365933493 container start 84e5b36bf2748fbb2b02457811c420c3982b92dd26d5661d886d40497d046692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:20:04.277 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:04 smithi012 podman[81318]: 2024-09-16 14:20:04.094432099 +0000 UTC m=+0.462406727 container attach 84e5b36bf2748fbb2b02457811c420c3982b92dd26d5661d886d40497d046692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=reef-e4e227b) 2024-09-16T14:20:04.277 INFO:journalctl@ceph.osd.1.smithi012.stdout:Sep 16 14:20:04 smithi012 podman[81318]: 2024-09-16 14:20:04.194236345 +0000 UTC m=+0.562210965 container died 84e5b36bf2748fbb2b02457811c420c3982b92dd26d5661d886d40497d046692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T14:20:04.809 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T14:20:04.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:04.855 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T14:20:04.855 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T14:20:04.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.2 2024-09-16T14:20:05.141 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:04 smithi012 systemd[1]: Stopping Ceph osd.2 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:05.526 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:05 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2[57114]: 2024-09-16T14:20:05.139+0000 7ffa8ae83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:05.527 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:05 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2[57114]: 2024-09-16T14:20:05.139+0000 7ffa8ae83640 -1 osd.2 72 *** Got signal Terminated *** 2024-09-16T14:20:05.527 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:05 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2[57114]: 2024-09-16T14:20:05.139+0000 7ffa8ae83640 -1 osd.2 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:10.777 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:10 smithi012 podman[81420]: 2024-09-16 14:20:10.46083605 +0000 UTC m=+5.420199553 container died 216f39fc85bd0ae5bb005530240ce8ef2e45f48da41ce8194155f94ab7295deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T14:20:11.277 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:10 smithi012 podman[81420]: 2024-09-16 14:20:10.872090909 +0000 UTC m=+5.831454408 container cleanup 216f39fc85bd0ae5bb005530240ce8ef2e45f48da41ce8194155f94ab7295deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T14:20:11.277 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:10 smithi012 bash[81420]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2 2024-09-16T14:20:11.277 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81435]: 2024-09-16 14:20:11.065300591 +0000 UTC m=+0.606622275 container remove 216f39fc85bd0ae5bb005530240ce8ef2e45f48da41ce8194155f94ab7295deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T14:20:11.777 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81523]: 2024-09-16 14:20:11.500004681 +0000 UTC m=+0.112566464 container create 052fbc3882b2252d0e12cbdfc8f9df80ca97c4b72ad84d2baa217b1fb0b7ebf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-e4e227b, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T14:20:11.777 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81523]: 2024-09-16 14:20:11.404210458 +0000 UTC m=+0.016772255 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:11.777 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81523]: 2024-09-16 14:20:11.710706315 +0000 UTC m=+0.323268098 container init 052fbc3882b2252d0e12cbdfc8f9df80ca97c4b72ad84d2baa217b1fb0b7ebf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=reef-e4e227b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:20:11.777 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81523]: 2024-09-16 14:20:11.714360269 +0000 UTC m=+0.326922051 container start 052fbc3882b2252d0e12cbdfc8f9df80ca97c4b72ad84d2baa217b1fb0b7ebf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T14:20:12.214 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81523]: 2024-09-16 14:20:11.785523737 +0000 UTC m=+0.398085535 container attach 052fbc3882b2252d0e12cbdfc8f9df80ca97c4b72ad84d2baa217b1fb0b7ebf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T14:20:12.214 INFO:journalctl@ceph.osd.2.smithi012.stdout:Sep 16 14:20:11 smithi012 podman[81523]: 2024-09-16 14:20:11.917092931 +0000 UTC m=+0.529654724 container died 052fbc3882b2252d0e12cbdfc8f9df80ca97c4b72ad84d2baa217b1fb0b7ebf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T14:20:12.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T14:20:12.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:12.654 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T14:20:12.654 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T14:20:12.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.3 2024-09-16T14:20:12.993 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:12 smithi012 systemd[1]: Stopping Ceph osd.3 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:13.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:12 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:20:12.991+0000 7f7c4f17c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:13.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:12 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:20:12.991+0000 7f7c4f17c640 -1 osd.3 72 *** Got signal Terminated *** 2024-09-16T14:20:13.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:12 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:20:12.991+0000 7f7c4f17c640 -1 osd.3 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:15.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:15 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:15.265+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:16.088 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:15 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:15.617+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:16.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:16 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:16.293+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:16.945 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:16 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:16.618+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:17.271 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:16 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:16.944+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:17.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:16 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:20:16.851+0000 7f7c47788640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:50.981195+0000 front 2024-09-16T14:19:50.981287+0000 (oldest deadline 2024-09-16T14:20:16.281260+0000) 2024-09-16T14:20:17.587 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:17 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:17.269+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:18.088 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:17 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:17.657+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:18.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:17 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:17.898+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:18.089 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:17 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:17.853+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:18.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:17 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3[64840]: 2024-09-16T14:20:17.881+0000 7f7c47788640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:50.981195+0000 front 2024-09-16T14:19:50.981287+0000 (oldest deadline 2024-09-16T14:20:16.281260+0000) 2024-09-16T14:20:18.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:18 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:18.226+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:18.777 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:18 smithi012 podman[81624]: 2024-09-16 14:20:18.446155291 +0000 UTC m=+5.560580617 container died 1cd6a60cff8be502d3ae992b37f6a9ffa29ee1da461d772087088d5463788ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=reef-e4e227b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T14:20:18.946 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:18 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:18.838+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:18.946 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:18 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:18.611+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:19.201 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:18 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:18.945+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:19.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:18 smithi012 podman[81624]: 2024-09-16 14:20:18.820288424 +0000 UTC m=+5.934713780 container cleanup 1cd6a60cff8be502d3ae992b37f6a9ffa29ee1da461d772087088d5463788ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T14:20:19.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:18 smithi012 bash[81624]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3 2024-09-16T14:20:19.277 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81641]: 2024-09-16 14:20:19.060188334 +0000 UTC m=+0.607143879 container remove 1cd6a60cff8be502d3ae992b37f6a9ffa29ee1da461d772087088d5463788ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:20:19.570 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:19 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:19.200+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:19.689 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81730]: 2024-09-16 14:20:19.369347368 +0000 UTC m=+0.091913298 container create bfd6bbaea358babc3f4cc5fff27cd01641d6c328f6ceeb4fd831fe149ea7b144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T14:20:19.689 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81730]: 2024-09-16 14:20:19.293794639 +0000 UTC m=+0.016360540 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:19.689 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81730]: 2024-09-16 14:20:19.605153683 +0000 UTC m=+0.327719586 container init bfd6bbaea358babc3f4cc5fff27cd01641d6c328f6ceeb4fd831fe149ea7b144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=reef-e4e227b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T14:20:19.689 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81730]: 2024-09-16 14:20:19.608881374 +0000 UTC m=+0.331447261 container start bfd6bbaea358babc3f4cc5fff27cd01641d6c328f6ceeb4fd831fe149ea7b144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T14:20:19.689 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81730]: 2024-09-16 14:20:19.688033033 +0000 UTC m=+0.410598924 container attach bfd6bbaea358babc3f4cc5fff27cd01641d6c328f6ceeb4fd831fe149ea7b144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True) 2024-09-16T14:20:19.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:19 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:19.569+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:20.027 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 16 14:20:19 smithi012 podman[81730]: 2024-09-16 14:20:19.806960929 +0000 UTC m=+0.529526826 container died bfd6bbaea358babc3f4cc5fff27cd01641d6c328f6ceeb4fd831fe149ea7b144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux ) 2024-09-16T14:20:20.172 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:19 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:19.903+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:20.172 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:19 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:19.876+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:20.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T14:20:20.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:20.516 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T14:20:20.516 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T14:20:20.516 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.4 2024-09-16T14:20:20.545 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:20.171+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:20.829 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:20 smithi189 systemd[1]: Stopping Ceph osd.4 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:20.829 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:20.553+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:21.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:20.832+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:21.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:20.827+0000 7f8dc0321640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:21.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:20.827+0000 7f8dc0321640 -1 osd.4 72 *** Got signal Terminated *** 2024-09-16T14:20:21.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:20.827+0000 7f8dc0321640 -1 osd.4 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:21.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:20 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:20.879+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:21.573 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:21.123+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:21.837 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:21.571+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:22.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:21.836+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:22.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:21 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:21.901+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:22.542 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:22 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:22.095+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:22.800 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:22 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:22.541+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:22.800 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:22 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:22.798+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:23.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:22 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:22.946+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:23.512 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:23.118+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:23.513 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:20:23.267Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.12:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.12:8765: connect: connection refused" 2024-09-16T14:20:23.513 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:20:23.267Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.12:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.12:8765: connect: connection refused" 2024-09-16T14:20:23.513 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:20:23.269Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.12:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.12:8765: connect: connection refused" 2024-09-16T14:20:23.828 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:23.508+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:24.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:23.897+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:24.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:23.827+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:24.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:23 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:23.827+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:24.472 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:24.093+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:24.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:24.471+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:25.338 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4[35272]: 2024-09-16T14:20:24.904+0000 7f8db892d640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.094565+0000 front 2024-09-16T14:19:51.094430+0000 (oldest deadline 2024-09-16T14:20:16.394355+0000) 2024-09-16T14:20:25.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:25.070+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:25.338 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:24.849+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:25.338 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:24 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:24.849+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:25.806 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:25.491+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:26.088 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:26.032+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:26.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:25.805+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:26.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:25 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:25.805+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:26.487 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:26 smithi189 podman[70579]: 2024-09-16 14:20:26.202592875 +0000 UTC m=+5.489765413 container died 57bf54c314d0cc810f3b97927b57dd15e52a09705f70a94b5157d4b1f0fa193e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T14:20:26.787 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:26 smithi189 podman[70579]: 2024-09-16 14:20:26.585702575 +0000 UTC m=+5.872875110 container cleanup 57bf54c314d0cc810f3b97927b57dd15e52a09705f70a94b5157d4b1f0fa193e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:20:26.788 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:26 smithi189 bash[70579]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4 2024-09-16T14:20:26.788 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:26.513+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:26.788 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:26.513+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:27.088 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:26 smithi189 podman[70594]: 2024-09-16 14:20:26.786843653 +0000 UTC m=+0.582087952 container remove 57bf54c314d0cc810f3b97927b57dd15e52a09705f70a94b5157d4b1f0fa193e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-e4e227b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T14:20:27.088 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:27.021+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:27.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:26.809+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:27.089 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:26 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:26.809+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:27.426 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:27 smithi189 podman[70682]: 2024-09-16 14:20:27.112685439 +0000 UTC m=+0.104201183 container create a01f21f1abd8969baeef32fec6c0f740cea7069620e23a3f59eaf284d660b7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T14:20:27.426 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:27 smithi189 podman[70682]: 2024-09-16 14:20:27.024177473 +0000 UTC m=+0.015693223 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:27.426 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:27 smithi189 podman[70682]: 2024-09-16 14:20:27.342258663 +0000 UTC m=+0.333774407 container init a01f21f1abd8969baeef32fec6c0f740cea7069620e23a3f59eaf284d660b7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS) 2024-09-16T14:20:27.426 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:27 smithi189 podman[70682]: 2024-09-16 14:20:27.345879799 +0000 UTC m=+0.337395555 container start a01f21f1abd8969baeef32fec6c0f740cea7069620e23a3f59eaf284d660b7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T14:20:27.426 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:27 smithi189 podman[70682]: 2024-09-16 14:20:27.425559564 +0000 UTC m=+0.417075320 container attach a01f21f1abd8969baeef32fec6c0f740cea7069620e23a3f59eaf284d660b7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T14:20:27.791 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 16 14:20:27 smithi189 podman[70682]: 2024-09-16 14:20:27.541198984 +0000 UTC m=+0.532714730 container died a01f21f1abd8969baeef32fec6c0f740cea7069620e23a3f59eaf284d660b7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:20:27.791 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:27.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:27.791 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:27.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:28.068 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:27.789+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:28.069 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:27 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:27.789+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:28.179 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T14:20:28.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:28.259 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T14:20:28.260 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T14:20:28.260 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.5 2024-09-16T14:20:28.320 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:28.067+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:28.588 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:28 smithi189 systemd[1]: Stopping Ceph osd.5 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:28.588 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:28.494+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:28.588 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:28.494+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:29.041 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:28.752+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:29.041 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:28.752+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:29.041 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:28.624+0000 7f323eeee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:29.041 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:28.624+0000 7f323eeee640 -1 osd.5 72 *** Got signal Terminated *** 2024-09-16T14:20:29.041 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:28 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:28.624+0000 7f323eeee640 -1 osd.5 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:29.041 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:29.037+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:29.042 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:29.037+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.342370+0000 front 2024-09-16T14:20:01.342164+0000 (oldest deadline 2024-09-16T14:20:28.342414+0000) 2024-09-16T14:20:29.756 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:29.513+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:29.757 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:29.513+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:30.022 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:29.755+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:30.023 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:29 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:29.755+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:30.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:30.021+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:30.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:30.021+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.342370+0000 front 2024-09-16T14:20:01.342164+0000 (oldest deadline 2024-09-16T14:20:28.342414+0000) 2024-09-16T14:20:30.762 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:30.501+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:30.762 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:30.501+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:30.762 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:30.501+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:31.070 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:30.761+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:31.070 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:30 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:30.761+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:31.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:31.069+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:31.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:31.069+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.342370+0000 front 2024-09-16T14:20:01.342164+0000 (oldest deadline 2024-09-16T14:20:28.342414+0000) 2024-09-16T14:20:31.741 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:31.472+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:31.741 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:31.472+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:31.741 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:31.472+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:32.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:31.739+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:32.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:31 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:31.739+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:32.512 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:32.090+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:32.512 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:32.090+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.342370+0000 front 2024-09-16T14:20:01.342164+0000 (oldest deadline 2024-09-16T14:20:28.342414+0000) 2024-09-16T14:20:32.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:32.731+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:32.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:32.731+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:32.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:32.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:32.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:32.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:32.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:32 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:32.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:33.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:33.057+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:52.640958+0000 front 2024-09-16T14:19:52.641014+0000 (oldest deadline 2024-09-16T14:20:14.940728+0000) 2024-09-16T14:20:33.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:33.057+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.342370+0000 front 2024-09-16T14:20:01.342164+0000 (oldest deadline 2024-09-16T14:20:28.342414+0000) 2024-09-16T14:20:33.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5[42477]: 2024-09-16T14:20:33.057+0000 7f32374fa640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.343161+0000 front 2024-09-16T14:20:08.343201+0000 (oldest deadline 2024-09-16T14:20:33.042884+0000) 2024-09-16T14:20:33.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:33.700+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:33.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:33.700+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:33.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:33.543+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:33.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:33.543+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:33.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:33 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:33.543+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:34.460 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:34 smithi189 podman[70784]: 2024-09-16 14:20:34.16598176 +0000 UTC m=+5.666627395 container died 8541f645d0810b38bda6112fbc70c5639fd95b966de53da8f740136f496e2335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=reef-e4e227b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T14:20:34.724 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:34 smithi189 podman[70784]: 2024-09-16 14:20:34.566102708 +0000 UTC m=+6.066748342 container cleanup 8541f645d0810b38bda6112fbc70c5639fd95b966de53da8f740136f496e2335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:20:34.724 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:34 smithi189 bash[70784]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5 2024-09-16T14:20:34.724 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:34 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:34.528+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:34.725 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:34 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:34.528+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:34.725 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:34 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:34.528+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:35.088 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:34 smithi189 podman[70811]: 2024-09-16 14:20:34.801210746 +0000 UTC m=+0.631850196 container remove 8541f645d0810b38bda6112fbc70c5639fd95b966de53da8f740136f496e2335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:20:35.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:34 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:34.721+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:35.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:34 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:34.721+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:35.413 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:35 smithi189 podman[70900]: 2024-09-16 14:20:35.037839193 +0000 UTC m=+0.016023057 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:35.413 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:35 smithi189 podman[70900]: 2024-09-16 14:20:35.135219504 +0000 UTC m=+0.113403407 container create fd507f21724974f39d092eaaa5235045e6aa029588c211c958c1ce9a7edfdd68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T14:20:35.413 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:35 smithi189 podman[70900]: 2024-09-16 14:20:35.345933014 +0000 UTC m=+0.324116878 container init fd507f21724974f39d092eaaa5235045e6aa029588c211c958c1ce9a7edfdd68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:20:35.413 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:35 smithi189 podman[70900]: 2024-09-16 14:20:35.349277938 +0000 UTC m=+0.327461809 container start fd507f21724974f39d092eaaa5235045e6aa029588c211c958c1ce9a7edfdd68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T14:20:35.413 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:35 smithi189 podman[70900]: 2024-09-16 14:20:35.412361825 +0000 UTC m=+0.390545696 container attach fd507f21724974f39d092eaaa5235045e6aa029588c211c958c1ce9a7edfdd68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T14:20:35.737 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 16 14:20:35 smithi189 podman[70900]: 2024-09-16 14:20:35.537328262 +0000 UTC m=+0.515512124 container died fd507f21724974f39d092eaaa5235045e6aa029588c211c958c1ce9a7edfdd68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T14:20:35.737 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:35.487+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:35.737 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:35.487+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:35.737 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:35.487+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:36.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:35.734+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:36.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:35.734+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:36.088 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:35 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:35.734+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:09.456219+0000 front 2024-09-16T14:20:09.456157+0000 (oldest deadline 2024-09-16T14:20:35.356027+0000) 2024-09-16T14:20:36.148 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T14:20:36.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:36.228 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T14:20:36.228 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T14:20:36.228 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.6 2024-09-16T14:20:36.506 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 systemd[1]: Stopping Ceph osd.6 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:36.504+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:36.504+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:36.504+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:36.551+0000 7f607ec37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:36.551+0000 7f607ec37640 -1 osd.6 72 *** Got signal Terminated *** 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:36.551+0000 7f607ec37640 -1 osd.6 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:36.701+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:36.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:36.701+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:36.839 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:36 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:36.701+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:09.456219+0000 front 2024-09-16T14:20:09.456157+0000 (oldest deadline 2024-09-16T14:20:35.356027+0000) 2024-09-16T14:20:37.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:37 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:37.705+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:37.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:37 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:37.705+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:37.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:37 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:37.705+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:09.456219+0000 front 2024-09-16T14:20:09.456157+0000 (oldest deadline 2024-09-16T14:20:35.356027+0000) 2024-09-16T14:20:37.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:37 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:37.544+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:37.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:37 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:37.544+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:37.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:37 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:37.544+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:38.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:38 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:38.660+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:38.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:38 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:38.660+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:38.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:38 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:38.660+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:09.456219+0000 front 2024-09-16T14:20:09.456157+0000 (oldest deadline 2024-09-16T14:20:35.356027+0000) 2024-09-16T14:20:38.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:38 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:38.515+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:38.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:38 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:38.515+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:38.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:38 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:38.515+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:39.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:39 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:39.678+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:39.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:39 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:39.678+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:39.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:39 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:39.678+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:09.456219+0000 front 2024-09-16T14:20:09.456157+0000 (oldest deadline 2024-09-16T14:20:35.356027+0000) 2024-09-16T14:20:39.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:39 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:39.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:39.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:39 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:39.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:39.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:39 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:39.510+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:40.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:40 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:40.689+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:51.453566+0000 front 2024-09-16T14:19:51.453800+0000 (oldest deadline 2024-09-16T14:20:17.353701+0000) 2024-09-16T14:20:40.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:40 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:40.689+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:19:59.054862+0000 front 2024-09-16T14:19:59.054746+0000 (oldest deadline 2024-09-16T14:20:23.154578+0000) 2024-09-16T14:20:40.838 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:40 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6[50244]: 2024-09-16T14:20:40.689+0000 7f6077243640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:09.456219+0000 front 2024-09-16T14:20:09.456157+0000 (oldest deadline 2024-09-16T14:20:35.356027+0000) 2024-09-16T14:20:40.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:40 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:40.555+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:40.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:40 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:40.555+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:40.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:40 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:40.555+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:41.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:41 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:41.569+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:41.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:41 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:41.569+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:41.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:41 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:41.569+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:42.338 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:42 smithi189 podman[71005]: 2024-09-16 14:20:42.074819959 +0000 UTC m=+5.625305033 container died 79ffa81f9451ddebd1da6e787695d17520d0493221bbfdd7cad8134c88085130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T14:20:42.695 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:42 smithi189 podman[71005]: 2024-09-16 14:20:42.427214402 +0000 UTC m=+5.977699474 container cleanup 79ffa81f9451ddebd1da6e787695d17520d0493221bbfdd7cad8134c88085130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T14:20:42.696 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:42 smithi189 bash[71005]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6 2024-09-16T14:20:42.696 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:42.588+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:42.696 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:42.588+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:42.696 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:42.588+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:42.696 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:42 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:42.588+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:43.038 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:42 smithi189 podman[71026]: 2024-09-16 14:20:42.694997822 +0000 UTC m=+0.621377335 container remove 79ffa81f9451ddebd1da6e787695d17520d0493221bbfdd7cad8134c88085130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T14:20:43.333 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:43 smithi189 podman[71116]: 2024-09-16 14:20:43.037730367 +0000 UTC m=+0.107235584 container create 2749a3732eba26796df2f1dcdab03c5f69aeaf55bf7ebf6711e901f30c07dc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:20:43.333 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:43 smithi189 podman[71116]: 2024-09-16 14:20:42.946152771 +0000 UTC m=+0.015657946 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:43.333 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:43 smithi189 podman[71116]: 2024-09-16 14:20:43.248488001 +0000 UTC m=+0.317993173 container init 2749a3732eba26796df2f1dcdab03c5f69aeaf55bf7ebf6711e901f30c07dc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T14:20:43.333 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:43 smithi189 podman[71116]: 2024-09-16 14:20:43.251736867 +0000 UTC m=+0.321242038 container start 2749a3732eba26796df2f1dcdab03c5f69aeaf55bf7ebf6711e901f30c07dc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T14:20:43.333 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:43 smithi189 podman[71116]: 2024-09-16 14:20:43.33186202 +0000 UTC m=+0.401367194 container attach 2749a3732eba26796df2f1dcdab03c5f69aeaf55bf7ebf6711e901f30c07dc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-e4e227b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T14:20:43.588 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 16 14:20:43 smithi189 podman[71116]: 2024-09-16 14:20:43.448883999 +0000 UTC m=+0.518389181 container died 2749a3732eba26796df2f1dcdab03c5f69aeaf55bf7ebf6711e901f30c07dc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T14:20:43.588 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:43 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:43.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:43.588 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:43 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:43.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:43.588 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:43 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:43.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:43.588 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:43 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:43.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:44.001 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T14:20:44.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:44.040 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T14:20:44.040 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T14:20:44.040 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.7 2024-09-16T14:20:44.338 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 systemd[1]: Stopping Ceph osd.7 for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:44.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.345+0000 7f6e3d1cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:44.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.345+0000 7f6e3d1cf640 -1 osd.7 72 *** Got signal Terminated *** 2024-09-16T14:20:44.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.345+0000 7f6e3d1cf640 -1 osd.7 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T14:20:44.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.593+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:44.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.593+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:44.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.593+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:44.839 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:44 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:44.593+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:46.088 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:45 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:45.605+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:46.088 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:45 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:45.605+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:46.088 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:45 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:45.605+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:46.088 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:45 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:45.605+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:46.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:46 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:46.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:46.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:46 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:46.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:46.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:46 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:46.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:46.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:46 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:46.564+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:47.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:47 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:47.516+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:47.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:47 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:47.516+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:47.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:47 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:47.516+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:47.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:47 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:47.516+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:47.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:47 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:47.516+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.189:6804 osd.4 since back 2024-09-16T14:20:23.871711+0000 front 2024-09-16T14:20:23.871628+0000 (oldest deadline 2024-09-16T14:20:47.371571+0000) 2024-09-16T14:20:48.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:48 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:48.489+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-16T14:19:49.466675+0000 front 2024-09-16T14:19:49.466692+0000 (oldest deadline 2024-09-16T14:20:14.766601+0000) 2024-09-16T14:20:48.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:48 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:48.489+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6814 osd.1 since back 2024-09-16T14:20:01.168453+0000 front 2024-09-16T14:20:01.168348+0000 (oldest deadline 2024-09-16T14:20:25.868202+0000) 2024-09-16T14:20:48.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:48 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:48.489+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6822 osd.2 since back 2024-09-16T14:20:08.169343+0000 front 2024-09-16T14:20:08.169305+0000 (oldest deadline 2024-09-16T14:20:30.469136+0000) 2024-09-16T14:20:48.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:48 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:48.489+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.12:6830 osd.3 since back 2024-09-16T14:20:15.770244+0000 front 2024-09-16T14:20:15.770221+0000 (oldest deadline 2024-09-16T14:20:42.170657+0000) 2024-09-16T14:20:48.838 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:48 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7[58696]: 2024-09-16T14:20:48.489+0000 7f6e357db640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.189:6804 osd.4 since back 2024-09-16T14:20:23.871711+0000 front 2024-09-16T14:20:23.871628+0000 (oldest deadline 2024-09-16T14:20:47.371571+0000) 2024-09-16T14:20:50.005 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:49 smithi189 podman[71218]: 2024-09-16 14:20:49.726765719 +0000 UTC m=+5.469960102 container died fa1a1f2f768396dc8bc4fc8b75c3af612d10cb776d5835fc25b532d21352caaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T14:20:50.338 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 podman[71218]: 2024-09-16 14:20:50.121266876 +0000 UTC m=+5.864461245 container cleanup fa1a1f2f768396dc8bc4fc8b75c3af612d10cb776d5835fc25b532d21352caaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T14:20:50.338 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 bash[71218]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7 2024-09-16T14:20:50.681 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 podman[71234]: 2024-09-16 14:20:50.372320842 +0000 UTC m=+0.644657176 container remove fa1a1f2f768396dc8bc4fc8b75c3af612d10cb776d5835fc25b532d21352caaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T14:20:51.010 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 podman[71321]: 2024-09-16 14:20:50.583395969 +0000 UTC m=+0.016431678 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T14:20:51.010 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 podman[71321]: 2024-09-16 14:20:50.689675327 +0000 UTC m=+0.122711036 container create 69d0693f1366a84bd4ef80966662efa51e71049624f6e14dc9ed4f22c687264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T14:20:51.010 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 podman[71321]: 2024-09-16 14:20:50.925583384 +0000 UTC m=+0.358619099 container init 69d0693f1366a84bd4ef80966662efa51e71049624f6e14dc9ed4f22c687264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T14:20:51.010 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:50 smithi189 podman[71321]: 2024-09-16 14:20:50.9290101 +0000 UTC m=+0.362045814 container start 69d0693f1366a84bd4ef80966662efa51e71049624f6e14dc9ed4f22c687264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T14:20:51.010 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:51 smithi189 podman[71321]: 2024-09-16 14:20:51.009030541 +0000 UTC m=+0.442066264 container attach 69d0693f1366a84bd4ef80966662efa51e71049624f6e14dc9ed4f22c687264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T14:20:51.338 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 16 14:20:51 smithi189 podman[71321]: 2024-09-16 14:20:51.123373965 +0000 UTC m=+0.556409679 container died 69d0693f1366a84bd4ef80966662efa51e71049624f6e14dc9ed4f22c687264f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T14:20:51.714 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T14:20:51.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:20:51.753 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T14:20:51.753 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-16T14:20:51.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-16T14:20:52.157 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:20:51 smithi012 systemd[1]: Stopping Ceph rgw.foo.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:20:52.527 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:20:52 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a[72103]: 2024-09-16T14:20:52.154+0000 7f77c08c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T14:20:52.527 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:20:52 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a[72103]: 2024-09-16T14:20:52.154+0000 7f77c4131880 -1 shutting down 2024-09-16T14:21:02.422 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:02 smithi012 bash[81832]: time="2024-09-16T14:21:02Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-16T14:21:02.422 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:02 smithi012 podman[81832]: 2024-09-16 14:21:02.17262603 +0000 UTC m=+10.208348828 container died ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T14:21:02.777 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:02 smithi012 podman[81832]: 2024-09-16 14:21:02.607674743 +0000 UTC m=+10.643397532 container cleanup ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T14:21:02.777 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:02 smithi012 bash[81832]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-16T14:21:03.119 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:02 smithi012 podman[81857]: 2024-09-16 14:21:02.881077257 +0000 UTC m=+0.702048567 container remove ac3b8513bf6c278dbf9314b875eea3f65d9cb2f9666e86f1868cfe9ef2427788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T14:21:03.119 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:02 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-16T14:21:03.119 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:03 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-16T14:21:03.120 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:03 smithi012 systemd[1]: Stopped Ceph rgw.foo.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:21:03.120 INFO:journalctl@ceph.rgw.foo.a.smithi012.stdout:Sep 16 14:21:03 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@rgw.foo.a.service: Consumed 1.510s CPU time. 2024-09-16T14:21:03.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-16T14:21:03.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:21:03.159 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-16T14:21:03.159 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-16T14:21:03.159 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-9a9578d0-7435-11ef-bceb-c7b262605968@prometheus.a 2024-09-16T14:21:03.456 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 systemd[1]: Stopping Ceph prometheus.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T14:21:03.732 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.455Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T14:21:03.733 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.456Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T14:21:03.733 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.456Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T14:21:03.733 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a[68068]: ts=2024-09-16T14:21:03.456Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T14:21:03.733 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 podman[71424]: 2024-09-16 14:21:03.466197155 +0000 UTC m=+0.111607322 container died 9f8db8f19abee196fe38e47cc73ccdaf2a59d6aed147c492cfdfbc43ef0e2dfc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:04.088 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 podman[71424]: 2024-09-16 14:21:03.914084606 +0000 UTC m=+0.559494761 container cleanup 9f8db8f19abee196fe38e47cc73ccdaf2a59d6aed147c492cfdfbc43ef0e2dfc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:04.088 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 16 14:21:03 smithi189 bash[71424]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T14:21:04.338 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a9578d0-7435-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-16T14:21:04.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T14:21:04.373 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-16T14:21:04.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T14:21:04.532 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:21:06.277 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:06 smithi012 systemd[1]: Stopping Ceph alertmanager.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:21:06.686 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:06 smithi012 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a[78604]: ts=2024-09-16T14:21:06.410Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T14:21:06.686 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:06 smithi012 podman[82289]: 2024-09-16 14:21:06.431508303 +0000 UTC m=+0.117282945 container died c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:07.027 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:06 smithi012 podman[82289]: 2024-09-16 14:21:06.776607261 +0000 UTC m=+0.462381893 container cleanup c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:07.027 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:06 smithi012 bash[82289]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T14:21:07.347 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:07 smithi012 podman[82303]: 2024-09-16 14:21:07.053108439 +0000 UTC m=+0.619265290 container remove c6ff5bda8ca828b22d221b9104dd1de699996b773e3b5b678bcf45c8a0a3439c (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:07.347 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:07 smithi012 podman[82303]: 2024-09-16 14:21:07.136867516 +0000 UTC m=+0.703024367 volume remove 90ad9464ad4d3710d01e75fcbbeff74831cd07c5319429dfa54ae6a3afd0e4b3 2024-09-16T14:21:07.777 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:07 smithi012 systemd[1]: Stopping Ceph node-exporter.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:21:07.777 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:07 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T14:21:07.777 INFO:journalctl@ceph.alertmanager.a.smithi012.stdout:Sep 16 14:21:07 smithi012 systemd[1]: Stopped Ceph alertmanager.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:21:08.132 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:07 smithi012 podman[82409]: 2024-09-16 14:21:07.842467591 +0000 UTC m=+0.130273478 container died dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:08.132 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 podman[82409]: 2024-09-16 14:21:08.131563215 +0000 UTC m=+0.419369104 container cleanup dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:08.383 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 bash[82409]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a 2024-09-16T14:21:08.681 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 podman[82422]: 2024-09-16 14:21:08.382580263 +0000 UTC m=+0.539319075 container remove dce327a696ae2072341bf7de7cf85a1726d20ff5f83a22cdd53257d84044d906 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T14:21:08.681 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T14:21:08.681 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-16T14:21:08.682 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 systemd[1]: Stopped Ceph node-exporter.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:21:08.682 INFO:journalctl@ceph.node-exporter.a.smithi012.stdout:Sep 16 14:21:08 smithi012 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 1.812s CPU time. 2024-09-16T14:21:09.198 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T14:21:09.351 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:21:10.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:10 smithi189 systemd[1]: Stopping Ceph iscsi.iscsi.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:21:11.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:10 smithi189 ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a[66460]: debug Shutdown received 2024-09-16T14:21:21.320 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:20 smithi189 bash[71807]: time="2024-09-16T14:21:20Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-16T14:21:21.320 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:20 smithi189 podman[71807]: 2024-09-16 14:21:20.928970134 +0000 UTC m=+10.124530575 container died 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-e4e227b) 2024-09-16T14:21:21.588 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:21 smithi189 podman[71807]: 2024-09-16 14:21:21.485051371 +0000 UTC m=+10.680611811 container cleanup 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T14:21:21.588 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:21 smithi189 bash[71807]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-16T14:21:22.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:21 smithi189 podman[71820]: 2024-09-16 14:21:21.761635292 +0000 UTC m=+0.832747532 container remove 4058dc7166f7529a02b1626109cd56d3a9ed0202019221a2dea06533f5641403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T14:21:22.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:21 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-16T14:21:22.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 podman[71866]: 2024-09-16 14:21:22.016303001 +0000 UTC m=+0.113258509 container died d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T14:21:22.700 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 podman[71866]: 2024-09-16 14:21:22.439137662 +0000 UTC m=+0.536093158 container cleanup d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b) 2024-09-16T14:21:22.700 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 bash[71866]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-16T14:21:22.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-16T14:21:22.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 66299 (conmon) remains running after unit stopped. 2024-09-16T14:21:22.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 71877 (podman) remains running after unit stopped. 2024-09-16T14:21:22.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 systemd[1]: Stopped Ceph iscsi.iscsi.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:21:22.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.155s CPU time. 2024-09-16T14:21:22.701 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 16 14:21:22 smithi189 podman[71877]: 2024-09-16 14:21:22.640575507 +0000 UTC m=+0.628017146 container remove d0a967fac412c99333a16f3155cc91f2ea3d6b2e7f0a61495b94e88fe8af4363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T14:21:23.088 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:22 smithi189 systemd[1]: Stopping Ceph grafana.a for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:21:23.412 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 podman[71997]: 2024-09-16 14:21:23.146880166 +0000 UTC m=+0.121786416 container died 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a, maintainer=Paul Cuzner , vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, architecture=x86_64, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, io.openshift.tags=base rhel8) 2024-09-16T14:21:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 podman[71997]: 2024-09-16 14:21:23.485411109 +0000 UTC m=+0.460317345 container cleanup 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, architecture=x86_64, io.buildah.version=1.32.0, name=ubi8, build-date=2022-04-29T11:48:11.055897) 2024-09-16T14:21:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 bash[71997]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a 2024-09-16T14:21:23.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 podman[72010]: 2024-09-16 14:21:23.661550264 +0000 UTC m=+0.512869898 container remove 668fb8189ce2eae496d91627f12babf22e82764db11e3ebef3f43ffdc8fb4293 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, version=8.5, maintainer=Paul Cuzner , io.buildah.version=1.32.0, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, architecture=x86_64, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, vendor=Red Hat, Inc., distribution-scope=public, release=239.1651231664, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-16T14:21:23.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T14:21:23.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-16T14:21:23.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 systemd[1]: Stopped Ceph grafana.a for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:21:23.916 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 16 14:21:23 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@grafana.a.service: Consumed 3.105s CPU time. 2024-09-16T14:21:24.334 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:24 smithi189 systemd[1]: Stopping Ceph node-exporter.b for 9a9578d0-7435-11ef-bceb-c7b262605968... 2024-09-16T14:21:24.588 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:24 smithi189 podman[72116]: 2024-09-16 14:21:24.333839645 +0000 UTC m=+0.108033615 container died 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T14:21:25.043 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:24 smithi189 podman[72116]: 2024-09-16 14:21:24.723001006 +0000 UTC m=+0.497194974 container cleanup 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T14:21:25.043 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:24 smithi189 bash[72116]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b 2024-09-16T14:21:25.043 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:24 smithi189 podman[72128]: 2024-09-16 14:21:24.940901639 +0000 UTC m=+0.611002091 container remove 959f2b5704de629c804d3c0d256c3132c8efaa55c790dab4a981445ec1d703c5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9a9578d0-7435-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T14:21:25.044 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:24 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-16T14:21:25.338 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:25 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-16T14:21:25.338 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:25 smithi189 systemd[1]: Stopped Ceph node-exporter.b for 9a9578d0-7435-11ef-bceb-c7b262605968. 2024-09-16T14:21:25.338 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 16 14:21:25 smithi189 systemd[1]: ceph-9a9578d0-7435-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 1.755s CPU time. 2024-09-16T14:21:25.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:21:25.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T14:21:25.787 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T14:21:25.789 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723/remote/smithi012/crash 2024-09-16T14:21:25.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/crash -- . 2024-09-16T14:21:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T14:21:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-09-16T14:21:25.847 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723/remote/smithi189/crash 2024-09-16T14:21:25.848 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/crash -- . 2024-09-16T14:21:25.886 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T14:21:25.887 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-09-16T14:21:25.887 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T14:21:25.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v CEPHADM_AGENT_DOWN | head -n 1 2024-09-16T14:21:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:grep: /var/log/ceph/9a9578d0-7435-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-16T14:21:25.921 INFO:tasks.cephadm:Compressing logs... 2024-09-16T14:21:25.921 DEBUG:teuthology.orchestra.run.smithi012:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:21:25.964 DEBUG:teuthology.orchestra.run.smithi189:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:21:25.988 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T14:21:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T14:21:27.467 INFO:tasks.cephadm:Archiving logs... 2024-09-16T14:21:27.467 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723/remote/smithi012/log 2024-09-16T14:21:27.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:21:27.715 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723/remote/smithi189/log 2024-09-16T14:21:27.716 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:21:27.851 INFO:tasks.cephadm:Removing cluster... 2024-09-16T14:21:27.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 --force 2024-09-16T14:21:27.995 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:21:28.128 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a9578d0-7435-11ef-bceb-c7b262605968 --force 2024-09-16T14:21:28.278 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 9a9578d0-7435-11ef-bceb-c7b262605968 2024-09-16T14:21:28.407 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T14:21:28.408 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T14:21:28.424 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T14:21:28.439 INFO:tasks.cephadm:Teardown complete 2024-09-16T14:21:28.439 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-16T14:21:28.449 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi012:/dev/vg_nvme/lv_1... 2024-09-16T14:21:28.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo nvme disconnect -n lv_1 2024-09-16T14:21:28.624 INFO:teuthology.orchestra.run.smithi012.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-16T14:21:28.625 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi012:/dev/vg_nvme/lv_2... 2024-09-16T14:21:28.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo nvme disconnect -n lv_2 2024-09-16T14:21:28.782 INFO:teuthology.orchestra.run.smithi012.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-16T14:21:28.784 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi012:/dev/vg_nvme/lv_3... 2024-09-16T14:21:28.784 DEBUG:teuthology.orchestra.run.smithi012:> sudo nvme disconnect -n lv_3 2024-09-16T14:21:28.920 INFO:teuthology.orchestra.run.smithi012.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-16T14:21:28.922 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi012:/dev/vg_nvme/lv_4... 2024-09-16T14:21:28.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo nvme disconnect -n lv_4 2024-09-16T14:21:29.074 INFO:teuthology.orchestra.run.smithi012.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-16T14:21:29.076 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T14:21:29.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/scratch_devs 2024-09-16T14:21:29.102 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_1... 2024-09-16T14:21:29.102 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_1 2024-09-16T14:21:29.249 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-16T14:21:29.251 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_2... 2024-09-16T14:21:29.251 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_2 2024-09-16T14:21:29.412 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-16T14:21:29.414 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_3... 2024-09-16T14:21:29.414 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_3 2024-09-16T14:21:29.624 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-16T14:21:29.626 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi189:/dev/vg_nvme/lv_4... 2024-09-16T14:21:29.626 DEBUG:teuthology.orchestra.run.smithi189:> sudo nvme disconnect -n lv_4 2024-09-16T14:21:29.808 INFO:teuthology.orchestra.run.smithi189.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-16T14:21:29.810 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-16T14:21:29.810 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/scratch_devs 2024-09-16T14:21:29.834 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T14:21:29.843 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T14:21:29.844 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:21:29.846 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:21:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:21:29.867 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:21:29.867 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-09-16T14:21:29.867 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +73us[ +84us] +/- 53ms 2024-09-16T14:21:29.867 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -183us[ -183us] +/- 69ms 2024-09-16T14:21:29.867 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +189us[ +189us] +/- 91ms 2024-09-16T14:21:29.867 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:21:29.889 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:21:29.893 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:21:29.893 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-16T14:21:29.893 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +44us[ +47us] +/- 53ms 2024-09-16T14:21:29.894 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -206us[ -206us] +/- 69ms 2024-09-16T14:21:29.894 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +175us[ +178us] +/- 91ms 2024-09-16T14:21:29.894 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:21:29.894 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T14:21:29.903 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T14:21:29.904 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T14:21:29.927 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T14:21:30.007 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T14:21:30.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T14:21:30.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:21:30.106 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T14:21:30.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:21:30.135 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T14:21:30.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T14:21:30.169 INFO:teuthology.task.internal:Duration was 821.538019 seconds 2024-09-16T14:21:30.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T14:21:30.204 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T14:21:30.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:21:30.207 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:21:30.246 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:21:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:21:30.625 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T14:21:30.625 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T14:21:30.625 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:21:30.649 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-16T14:21:30.649 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:21:30.692 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T14:21:30.693 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:21:30.695 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:21:30.760 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T14:21:30.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:21:30.766 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:21:31.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T14:21:31.053 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T14:21:31.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:21:31.078 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:21:31.104 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T14:21:31.113 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:21:31.121 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:21:31.143 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-09-16T14:21:31.167 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-16T14:21:31.191 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:21:31.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:21:31.218 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:21:31.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:21:31.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T14:21:31.255 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T14:21:31.256 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723/remote/smithi012 2024-09-16T14:21:31.256 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:21:31.298 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906723/remote/smithi189 2024-09-16T14:21:31.299 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:21:31.335 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T14:21:31.335 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:21:31.337 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:21:31.384 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T14:21:31.395 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T14:21:31.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T14:21:31.412 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T14:21:31.412 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:21:31.414 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:21:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:21 /home/ubuntu/cephtest 2024-09-16T14:21:31.439 INFO:teuthology.orchestra.run.smithi189.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:21 /home/ubuntu/cephtest 2024-09-16T14:21:31.440 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T14:21:31.449 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T14:21:31.514 INFO:teuthology.run:Summary data: description: rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 821.5380194187164 owner: scheduled_teuthology@teuthology success: true 2024-09-16T14:21:31.514 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:21:31.603 INFO:teuthology.run:pass