2024-09-16T06:05:48.389 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T06:05:48.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T06:05:48.491 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536 branch: reef description: rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906536' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - CEPHADM_AGENT_DOWN log-only-match: - CEPHADM_ sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLGR9eq6/gsxBF0ZWb7ASjpf8OUkU85cskFdOpJgOB7cqbS17f/Z/pQ0anUn3x9bWIXOE3gMDy1fZEWKB6XpVyo= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1p/7VhAzRP+5f+NDGyuX6uxpVV+EZpkpQu+BLCPSaYf6Cn0FOofqvraIeRovMSe4JdKTsTHJbRiY0FCmPsMB8= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T06:05:48.491 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T06:05:48.492 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T06:05:48.492 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T06:05:48.494 INFO:teuthology.task.internal:Checking packages... 2024-09-16T06:05:48.515 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T06:05:48.515 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T06:05:48.515 INFO:teuthology.packaging:ref: None 2024-09-16T06:05:48.516 INFO:teuthology.packaging:tag: None 2024-09-16T06:05:48.516 INFO:teuthology.packaging:branch: reef 2024-09-16T06:05:48.516 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:05:48.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T06:05:48.763 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853.gae7f3733 2024-09-16T06:05:48.764 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T06:05:48.775 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T06:05:48.775 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T06:05:48.795 INFO:teuthology.task.internal:Saving configuration 2024-09-16T06:05:48.806 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T06:05:48.816 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T06:05:48.838 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 06:00:50.574692', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLGR9eq6/gsxBF0ZWb7ASjpf8OUkU85cskFdOpJgOB7cqbS17f/Z/pQ0anUn3x9bWIXOE3gMDy1fZEWKB6XpVyo='} 2024-09-16T06:05:48.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 06:00:50.572726', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1p/7VhAzRP+5f+NDGyuX6uxpVV+EZpkpQu+BLCPSaYf6Cn0FOofqvraIeRovMSe4JdKTsTHJbRiY0FCmPsMB8='} 2024-09-16T06:05:48.853 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T06:05:48.860 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T06:05:48.860 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-16T06:05:48.860 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T06:05:48.910 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc02ecb3ac0>, signals=[15]) 2024-09-16T06:05:48.911 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T06:05:48.917 INFO:teuthology.task.internal:Opening connections... 2024-09-16T06:05:48.917 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-09-16T06:05:48.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:05:48.999 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-16T06:05:48.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:05:49.075 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T06:05:49.082 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-09-16T06:05:49.100 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-09-16T06:05:49.101 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-09-16T06:05:49.159 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-09-16T06:05:49.159 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-09-16T06:05:49.159 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-09-16T06:05:49.159 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T06:05:49.160 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T06:05:49.161 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-09-16T06:05:49.184 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-16T06:05:49.202 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-16T06:05:49.203 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-16T06:05:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-16T06:05:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-16T06:05:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-16T06:05:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-16T06:05:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T06:05:49.260 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T06:05:49.260 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-16T06:05:49.276 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T06:05:49.285 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T06:05:49.289 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T06:05:49.289 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-09-16T06:05:49.292 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-16T06:05:49.315 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T06:05:49.322 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T06:05:49.322 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-09-16T06:05:49.349 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-16T06:05:49.431 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T06:05:49.483 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T06:05:49.505 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T06:05:49.505 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T06:05:49.505 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T06:05:49.506 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T06:05:49.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:05:49.523 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-16T06:05:49.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:05:49.524 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-16T06:05:49.581 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-503.el9.x86_64 2024-09-16T06:05:49.582 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-503.el9.x86_64 2024-09-16T06:05:49.582 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-09-16T06:05:49.583 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-503.el9.x86_64 2024-09-16T06:05:49.583 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-503.el9.x86_64 2024-09-16T06:05:49.583 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-09-16T06:05:51.937 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-16T06:05:51.981 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-16T06:05:57.081 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 20 MB 00:04 2024-09-16T06:06:00.921 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-16T06:06:01.456 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T06:06:02.832 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 113 kB/s | 19 kB 00:00 2024-09-16T06:06:03.786 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-16T06:06:06.871 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-16T06:06:09.400 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-16T06:06:10.888 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 6.6 kB/s | 19 kB 00:02 2024-09-16T06:06:11.381 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T06:06:11.381 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T06:06:11.471 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-16T06:06:11.477 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-09-16T06:06:11.478 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:06:11.479 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 79 M 2024-09-16T06:06:11.479 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 126 M 2024-09-16T06:06:11.479 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-16T06:06:11.776 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T06:06:12.059 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 75 kB/s | 23 kB 00:00 2024-09-16T06:06:17.223 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T06:06:19.238 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T06:06:19.238 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T06:06:19.319 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-16T06:06:19.329 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:06:19.329 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-09-16T06:06:19.329 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:06:19.329 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-09-16T06:06:19.330 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:06:19.331 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 79 M 2024-09-16T06:06:19.331 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 126 M 2024-09-16T06:06:19.331 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-16T06:06:19.657 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 282 kB/s | 23 kB 00:00 2024-09-16T06:06:19.858 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.7 MB/s | 29 MB 00:07 2024-09-16T06:06:20.700 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-16T06:06:20.774 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-16T06:06:20.959 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.7 MB/s | 16 MB 00:09 2024-09-16T06:06:20.959 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-16T06:06:20.959 INFO:teuthology.orchestra.run.smithi196.stdout:Total 8.3 MB/s | 79 MB 00:09 2024-09-16T06:06:21.600 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.7 MB/s | 16 MB 00:02 2024-09-16T06:06:21.637 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-16T06:06:21.987 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-16T06:06:21.987 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-16T06:06:22.944 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-16T06:06:22.944 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-16T06:06:25.411 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-16T06:06:26.162 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T06:06:26.244 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T06:06:27.233 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.8 MB/s | 29 MB 00:07 2024-09-16T06:06:27.233 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-16T06:06:27.234 INFO:teuthology.orchestra.run.smithi190.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-16T06:06:27.915 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-16T06:06:28.277 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-16T06:06:28.277 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-16T06:06:29.058 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T06:06:29.195 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T06:06:29.233 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-16T06:06:29.234 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-16T06:06:31.692 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-16T06:06:32.359 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T06:06:32.441 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T06:06:34.016 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T06:06:34.062 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:06:35.216 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T06:06:35.344 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T06:06:38.913 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:06:40.155 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T06:06:40.244 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:06:45.074 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:17.196 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:17.241 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:17.659 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:17.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T06:07:17.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T06:07:17.660 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T06:07:18.515 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:18.515 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:07:18.515 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-16T06:07:18.516 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T06:07:18.516 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T06:07:18.516 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T06:07:18.516 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T06:07:18.516 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:07:18.516 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-16T06:07:18.855 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T06:07:18.948 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T06:07:18.949 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T06:07:18.949 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T06:07:18.949 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T06:07:18.949 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T06:07:18.950 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T06:07:18.950 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T06:07:18.950 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T06:07:19.009 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T06:07:19.009 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T06:07:19.009 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-09-16T06:07:19.049 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 82.el9 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8274670 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T06:07:19.050 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:07:19.051 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T06:07:19.052 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T06:07:19.052 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-505.el9.x86_64 2024-09-16T06:07:19.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T06:07:19.756 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-09-16T06:07:21.118 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T06:07:21.134 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-09-16T06:07:21.145 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-16T06:07:21.161 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.BCvU2zjXFK 2024-09-16T06:07:21.162 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BCvU2zjXFK 2024-09-16T06:07:21.230 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.BCvU2zjXFK 2024-09-16T06:07:21.367 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.BCvU2zjXFK is 6KB 2024-09-16T06:07:21.415 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.BCvU2zjXFK 2024-09-16T06:07:21.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T06:07:21.500 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T06:07:21.500 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T06:07:21.500 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T06:07:21.500 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T06:07:21.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T06:07:21.652 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-09-16T06:07:23.160 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:23.204 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:23.612 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:23.612 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T06:07:23.612 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T06:07:23.612 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T06:07:24.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:07:24.477 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-16T06:07:24.833 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T06:07:24.923 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T06:07:24.924 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T06:07:24.924 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T06:07:24.924 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T06:07:24.924 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T06:07:24.924 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T06:07:24.924 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T06:07:24.924 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T06:07:24.982 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T06:07:24.983 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T06:07:24.983 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-09-16T06:07:25.015 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-09-16T06:07:25.015 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 82.el9 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8274670 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-09-16T06:07:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:07:25.017 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T06:07:25.018 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T06:07:25.018 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-505.el9.x86_64 2024-09-16T06:07:25.018 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T06:07:25.677 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-09-16T06:07:27.035 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T06:07:27.070 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-09-16T06:07:27.072 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-16T06:07:27.087 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.Rkg66f4MXp 2024-09-16T06:07:27.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Rkg66f4MXp 2024-09-16T06:07:27.154 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.Rkg66f4MXp 2024-09-16T06:07:27.303 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.Rkg66f4MXp is 6KB 2024-09-16T06:07:27.352 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.Rkg66f4MXp 2024-09-16T06:07:27.367 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T06:07:27.430 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T06:07:27.430 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T06:07:27.430 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T06:07:27.430 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T06:07:27.431 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T06:07:27.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-09-16T06:07:51.677 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T06:07:51.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-16T06:07:51.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:07:57.593 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T06:07:57.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T06:07:57.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:08:26.177 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-16T06:08:35.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-16T06:08:35.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:08:53.569 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-16T06:08:57.597 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T06:09:05.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-16T06:09:05.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:09:05.919 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-16T06:09:06.319 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-16T06:09:06.319 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T06:09:06.319 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-16T06:09:06.334 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-505.el9.x86_64 2024-09-16T06:09:06.335 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T06:09:06.335 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T06:09:06.335 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T06:09:06.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T06:09:06.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:09:06.962 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-16T06:09:07.300 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T06:09:07.300 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T06:09:07.301 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-16T06:09:07.315 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-505.el9.x86_64 2024-09-16T06:09:07.315 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T06:09:07.316 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T06:09:07.316 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T06:09:07.336 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T06:09:07.337 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T06:09:07.422 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-09-16T06:09:07.448 DEBUG:teuthology.parallel:result is None 2024-09-16T06:09:08.316 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T06:09:08.316 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T06:09:08.385 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-09-16T06:09:08.424 DEBUG:teuthology.parallel:result is None 2024-09-16T06:09:08.425 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T06:09:08.432 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T06:09:08.432 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T06:09:08.434 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T06:09:08.450 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T06:09:08.456 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T06:09:08.466 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T06:09:08.466 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T06:09:08.495 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T06:09:08.518 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T06:09:08.524 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T06:09:08.524 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T06:09:08.555 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T06:09:08.586 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:09:08.592 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:09:08.596 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:09:08.603 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:09:08.606 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T06:09:08.615 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T06:09:08.615 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T06:09:08.645 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T06:09:08.670 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T06:09:08.682 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T06:09:08.683 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T06:09:08.714 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T06:09:08.729 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:09:08.815 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:09:08.865 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:09:08.949 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:09:08.994 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:09:08.994 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T06:09:09.065 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:09:09.109 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:09:09.154 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:09:09.236 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:09:09.274 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:09:09.274 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T06:09:09.338 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-09-16T06:09:09.340 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-16T06:09:09.386 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T06:09:09.421 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T06:09:09.957 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T06:09:09.964 INFO:teuthology.task.internal:Starting timer... 2024-09-16T06:09:09.964 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T06:09:10.063 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T06:09:10.074 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-16T06:09:10.074 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-09-16T06:09:10.123 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-09-16T06:09:10.125 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-16T06:09:10.196 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-16T06:09:10.198 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T06:09:10.198 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-09-16T06:09:10.223 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-09-16T06:09:10.223 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-16T06:09:10.264 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-16T06:09:10.265 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi190.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-16T06:09:10.265 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T06:09:10.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:09:10.292 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T06:09:10.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:09:10.330 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T06:09:10.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-09-16T06:09:10.365 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-16T06:09:10.405 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T06:09:10.414 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T06:09:10.427 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T06:09:10.428 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi190.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T06:12:55.165 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi190.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-16T06:12:55.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T06:12:55.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:12:55.244 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-16T06:12:55.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-16T06:12:55.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-16T06:12:55.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:12:55.348 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-16T06:12:55.423 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-16T06:12:55.423 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T06:12:55.437 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T06:12:55.437 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T06:12:55.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:12:55.441 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T06:12:55.441 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:12:55.467 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T06:12:55.481 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T06:12:55.492 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T06:12:55.506 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T06:12:55.507 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-09-16T06:12:55.517 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-09-16T06:12:55.530 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-16T06:12:55.532 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T06:12:55.541 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-16T06:12:55.546 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T06:12:55.555 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T06:12:55.570 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T06:12:55.601 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-16T06:12:55.605 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T06:12:55.605 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-16T06:12:55.605 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.605 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.606 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.606 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.632 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-16T06:12:55.635 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T06:12:55.635 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-16T06:12:55.635 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.635 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.635 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.636 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:12:55.637 INFO:teuthology.run_tasks:Running task pexec... 2024-09-16T06:12:55.647 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-16T06:12:55.647 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi190.front.sepia.ceph.com 2024-09-16T06:12:55.648 DEBUG:teuthology.orchestra.run.smithi190:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T06:12:55.648 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi196.front.sepia.ceph.com 2024-09-16T06:12:55.648 DEBUG:teuthology.orchestra.run.smithi196:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T06:12:55.999 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-16T06:12:56.026 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-16T06:12:56.026 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:12:56.026 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-09-16T06:12:56.026 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:12:56.026 INFO:teuthology.orchestra.run.smithi190.stdout:Removing: 2024-09-16T06:12:56.026 INFO:teuthology.orchestra.run.smithi190.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout:Removing unused dependencies: 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout:Remove 2 Packages 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout:Freed space: 5.5 M 2024-09-16T06:12:56.027 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout:Removing: 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout:Removing unused dependencies: 2024-09-16T06:12:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout:Remove 2 Packages 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout:Freed space: 5.5 M 2024-09-16T06:12:56.029 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-16T06:12:56.030 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-16T06:12:56.030 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-16T06:12:56.032 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-16T06:12:56.032 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-16T06:12:56.096 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-16T06:12:56.096 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-16T06:12:56.099 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-16T06:12:56.099 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-16T06:12:56.238 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-16T06:12:56.242 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-16T06:12:56.309 INFO:teuthology.orchestra.run.smithi190.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T06:12:56.322 INFO:teuthology.orchestra.run.smithi196.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T06:12:56.364 INFO:teuthology.orchestra.run.smithi190.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T06:12:56.375 INFO:teuthology.orchestra.run.smithi196.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T06:12:56.844 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T06:12:56.844 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T06:12:56.889 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T06:12:56.889 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T06:12:57.074 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T06:12:57.074 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:12:57.074 INFO:teuthology.orchestra.run.smithi190.stdout:Removed: 2024-09-16T06:12:57.074 INFO:teuthology.orchestra.run.smithi190.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T06:12:57.075 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:12:57.075 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-16T06:12:57.155 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T06:12:57.155 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:12:57.155 INFO:teuthology.orchestra.run.smithi196.stdout:Removed: 2024-09-16T06:12:57.156 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T06:12:57.156 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:12:57.156 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-16T06:12:57.897 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 06:12:09 AM UTC. 2024-09-16T06:12:57.967 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 06:12:09 AM UTC. 2024-09-16T06:12:58.032 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T06:12:58.033 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T06:12:58.034 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:12:58.034 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-16T06:12:58.034 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-16T06:12:58.034 INFO:teuthology.orchestra.run.smithi190.stdout:Install 7 Packages 2024-09-16T06:12:58.034 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:12:58.047 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 5.2 M 2024-09-16T06:12:58.047 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 19 M 2024-09-16T06:12:58.047 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-16T06:12:58.107 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T06:12:58.108 INFO:teuthology.orchestra.run.smithi196.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout:Install 7 Packages 2024-09-16T06:12:58.109 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:12:58.115 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 5.2 M 2024-09-16T06:12:58.115 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 19 M 2024-09-16T06:12:58.115 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-16T06:12:58.287 INFO:teuthology.orchestra.run.smithi190.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 414 kB/s | 44 kB 00:00 2024-09-16T06:12:58.338 INFO:teuthology.orchestra.run.smithi190.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 673 kB/s | 106 kB 00:00 2024-09-16T06:12:58.363 INFO:teuthology.orchestra.run.smithi190.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 952 kB/s | 72 kB 00:00 2024-09-16T06:12:58.388 INFO:teuthology.orchestra.run.smithi190.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-16T06:12:58.473 INFO:teuthology.orchestra.run.smithi196.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 499 kB/s | 44 kB 00:00 2024-09-16T06:12:58.498 INFO:teuthology.orchestra.run.smithi196.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 927 kB/s | 106 kB 00:00 2024-09-16T06:12:58.524 INFO:teuthology.orchestra.run.smithi196.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.4 MB/s | 72 kB 00:00 2024-09-16T06:12:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-16T06:12:58.555 INFO:teuthology.orchestra.run.smithi190.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.4 MB/s | 906 kB 00:00 2024-09-16T06:12:58.632 INFO:teuthology.orchestra.run.smithi190.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.1 MB/s | 837 kB 00:00 2024-09-16T06:12:58.683 INFO:teuthology.orchestra.run.smithi196.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.0 MB/s | 906 kB 00:00 2024-09-16T06:12:58.716 INFO:teuthology.orchestra.run.smithi196.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.3 MB/s | 837 kB 00:00 2024-09-16T06:12:58.858 INFO:teuthology.orchestra.run.smithi196.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 10 MB/s | 3.2 MB 00:00 2024-09-16T06:12:58.859 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-16T06:12:58.859 INFO:teuthology.orchestra.run.smithi196.stdout:Total 7.0 MB/s | 5.2 MB 00:00 2024-09-16T06:12:58.968 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-16T06:12:58.983 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-16T06:12:58.983 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-16T06:12:59.307 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-16T06:12:59.307 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-16T06:12:59.606 INFO:teuthology.orchestra.run.smithi190.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.6 MB/s | 3.2 MB 00:01 2024-09-16T06:12:59.606 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-16T06:12:59.606 INFO:teuthology.orchestra.run.smithi190.stdout:Total 3.3 MB/s | 5.2 MB 00:01 2024-09-16T06:12:59.716 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-16T06:12:59.730 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-16T06:12:59.731 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-16T06:12:59.760 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-16T06:12:59.852 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T06:12:59.895 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T06:12:59.924 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T06:13:00.055 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-16T06:13:00.055 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-16T06:13:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T06:13:00.172 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T06:13:00.506 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-16T06:13:00.600 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T06:13:00.623 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T06:13:00.643 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T06:13:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T06:13:00.671 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T06:13:00.761 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T06:13:00.829 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T06:13:00.877 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T06:13:00.920 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T06:13:01.352 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T06:13:01.381 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T06:13:01.499 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T06:13:01.568 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T06:13:01.865 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:13:02.215 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-16T06:13:02.518 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T06:13:02.518 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T06:13:02.518 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T06:13:02.518 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T06:13:02.518 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T06:13:02.519 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T06:13:02.519 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T06:13:02.609 DEBUG:teuthology.parallel:result is None 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T06:13:02.853 INFO:teuthology.orchestra.run.smithi190.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T06:13:02.854 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:13:02.854 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-16T06:13:03.253 DEBUG:teuthology.parallel:result is None 2024-09-16T06:13:03.253 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-16T06:13:03.263 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-16T06:13:03.263 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:13:03.263 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:13:03.279 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T06:13:03.279 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-09-16T06:13:03.334 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:13:01.923566798 +0000 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:01.460579756 +0000 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:01.460579756 +0000 2024-09-16T06:13:03.335 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:01.460579756 +0000 2024-09-16T06:13:03.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T06:13:03.398 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:13:03.398 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:13:03.398 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00020947 s, 2.4 MB/s 2024-09-16T06:13:03.399 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T06:13:03.456 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-09-16T06:13:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T06:13:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:03.532 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-16T06:13:03.532 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:03.532 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:03.532 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:13:01.923566798 +0000 2024-09-16T06:13:03.533 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:01.459579785 +0000 2024-09-16T06:13:03.533 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:01.459579785 +0000 2024-09-16T06:13:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:01.459579785 +0000 2024-09-16T06:13:03.533 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T06:13:03.596 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:13:03.596 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:13:03.596 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000195301 s, 2.6 MB/s 2024-09-16T06:13:03.597 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T06:13:03.653 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-09-16T06:13:03.708 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T06:13:03.708 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:03.708 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-16T06:13:03.709 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:03.709 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:03.709 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:13:01.924566770 +0000 2024-09-16T06:13:03.709 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:01.460579756 +0000 2024-09-16T06:13:03.709 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:01.460579756 +0000 2024-09-16T06:13:03.709 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:01.460579756 +0000 2024-09-16T06:13:03.709 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T06:13:03.771 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:13:03.771 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:13:03.772 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000214158 s, 2.4 MB/s 2024-09-16T06:13:03.773 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T06:13:03.829 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:13:01.924566770 +0000 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:01.458579812 +0000 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:01.458579812 +0000 2024-09-16T06:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:01.458579812 +0000 2024-09-16T06:13:03.885 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T06:13:03.949 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:13:03.949 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:13:03.949 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00020465 s, 2.5 MB/s 2024-09-16T06:13:03.950 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T06:13:04.006 DEBUG:teuthology.orchestra.run.smithi190:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-16T06:13:04.182 INFO:teuthology.orchestra.run.smithi190.stdout:loop 2024-09-16T06:13:04.184 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_1... 2024-09-16T06:13:04.184 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-16T06:13:04.217 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-16T06:13:04.252 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_11 2024-09-16T06:13:04.287 INFO:teuthology.orchestra.run.smithi190.stdout:connecting to device: nvme1 2024-09-16T06:13:04.292 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_2... 2024-09-16T06:13:04.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-16T06:13:04.367 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-16T06:13:04.399 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_21 2024-09-16T06:13:04.430 INFO:teuthology.orchestra.run.smithi190.stdout:connecting to device: nvme2 2024-09-16T06:13:04.432 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_3... 2024-09-16T06:13:04.432 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-16T06:13:04.507 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-16T06:13:04.539 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_31 2024-09-16T06:13:04.575 INFO:teuthology.orchestra.run.smithi190.stdout:connecting to device: nvme3 2024-09-16T06:13:04.578 INFO:tasks.nvme_loop:Connecting nvme_loop smithi190:/dev/vg_nvme/lv_4... 2024-09-16T06:13:04.578 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-16T06:13:04.648 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-16T06:13:04.679 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/vg_nvme/lv_41 2024-09-16T06:13:04.711 INFO:teuthology.orchestra.run.smithi190.stdout:connecting to device: nvme4 2024-09-16T06:13:04.713 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:13:04.713 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:13:04.769 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme list -o json 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "Devices":[ 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "NameSpace":1, 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "GenericPath":"/dev/ng4n1", 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "SerialNumber":"0981ec7af37c38d54881", 2024-09-16T06:13:04.836 INFO:teuthology.orchestra.run.smithi190.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "SectorSize":512 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "NameSpace":1, 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "GenericPath":"/dev/ng3n1", 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "SerialNumber":"185c1f0867c6621424b6", 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:04.837 INFO:teuthology.orchestra.run.smithi190.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "SectorSize":512 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "NameSpace":1, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "GenericPath":"/dev/ng2n1", 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "SerialNumber":"88f2efbd2595ffef68c9", 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:04.838 INFO:teuthology.orchestra.run.smithi190.stdout: "SectorSize":512 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "NameSpace":1, 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "GenericPath":"/dev/ng1n1", 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "SerialNumber":"82855bf38a6bbf57e11a", 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: "SectorSize":512 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-16T06:13:04.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "NameSpace":1, 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "GenericPath":"/dev/ng0n1", 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "Firmware":"8DV101H0", 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "SerialNumber":"CVFT63210012400BGN", 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "UsedBytes":400088457216, 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "MaximumLBA":781422768, 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "PhysicalSize":400088457216, 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: "SectorSize":512 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-16T06:13:04.840 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T06:13:04.841 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T06:13:04.842 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:13:04.842 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/scratch_devs 2024-09-16T06:13:04.906 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:13:04.906 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:13:04.926 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T06:13:04.927 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 965 Links: 1 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:13:01.083925517 +0000 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:00.726781178 +0000 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:00.726781178 +0000 2024-09-16T06:13:04.985 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:00.726781178 +0000 2024-09-16T06:13:04.986 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T06:13:05.054 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:13:05.054 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:13:05.055 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000298718 s, 1.7 MB/s 2024-09-16T06:13:05.056 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T06:13:05.116 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-16T06:13:05.173 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:13:01.083925517 +0000 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:00.727781582 +0000 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:00.727781582 +0000 2024-09-16T06:13:05.174 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:00.727781582 +0000 2024-09-16T06:13:05.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T06:13:05.240 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:13:05.241 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:13:05.241 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000204513 s, 2.5 MB/s 2024-09-16T06:13:05.242 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T06:13:05.302 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:13:01.083925517 +0000 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:00.727781582 +0000 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:00.727781582 +0000 2024-09-16T06:13:05.359 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:00.727781582 +0000 2024-09-16T06:13:05.360 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T06:13:05.424 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:13:05.424 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:13:05.425 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000197196 s, 2.6 MB/s 2024-09-16T06:13:05.426 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T06:13:05.483 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-16T06:13:05.539 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:13:01.083925517 +0000 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:00.728781986 +0000 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:00.728781986 +0000 2024-09-16T06:13:05.540 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:00.728781986 +0000 2024-09-16T06:13:05.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T06:13:05.604 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:13:05.605 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:13:05.605 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000205062 s, 2.5 MB/s 2024-09-16T06:13:05.606 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T06:13:05.664 DEBUG:teuthology.orchestra.run.smithi196:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-16T06:13:05.801 INFO:teuthology.orchestra.run.smithi196.stdout:loop 2024-09-16T06:13:05.802 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_1... 2024-09-16T06:13:05.802 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-16T06:13:05.835 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-16T06:13:05.865 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_11 2024-09-16T06:13:05.899 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme1 2024-09-16T06:13:05.901 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_2... 2024-09-16T06:13:05.901 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-16T06:13:05.976 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-16T06:13:06.007 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_21 2024-09-16T06:13:06.038 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme2 2024-09-16T06:13:06.041 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_3... 2024-09-16T06:13:06.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-16T06:13:06.115 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-16T06:13:06.146 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_31 2024-09-16T06:13:06.182 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme3 2024-09-16T06:13:06.184 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_4... 2024-09-16T06:13:06.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-16T06:13:06.259 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-16T06:13:06.289 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_41 2024-09-16T06:13:06.325 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme4 2024-09-16T06:13:06.327 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:13:06.327 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:13:06.383 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme list -o json 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout: "Devices":[ 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng4n1", 2024-09-16T06:13:06.453 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"6a05bd32c428f2e4cf4c", 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng3n1", 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:06.454 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"78bbd8eda7b9cc78ac1e", 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng2n1", 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:06.455 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"f507015bf10bb318bc61", 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng1n1", 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"9de659ba9459ef1eec13", 2024-09-16T06:13:06.456 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng0n1", 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"8DV101H0", 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"CVFT6246000Z400BGN", 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":400088457216, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":781422768, 2024-09-16T06:13:06.457 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":400088457216, 2024-09-16T06:13:06.458 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-16T06:13:06.458 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-16T06:13:06.458 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-16T06:13:06.458 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-16T06:13:06.459 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T06:13:06.459 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:13:06.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/scratch_devs 2024-09-16T06:13:06.525 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T06:13:06.620 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'CEPHADM_AGENT_DOWN'], 'log-only-match': ['CEPHADM_'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T06:13:06.621 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:13:06.621 INFO:tasks.cephadm:Cluster fsid is bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:13:06.621 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T06:13:06.622 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.190', 'mon.c': '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'mon.b': '172.21.15.196'} 2024-09-16T06:13:06.622 INFO:tasks.cephadm:First mon is mon.a on smithi190 2024-09-16T06:13:06.622 INFO:tasks.cephadm:First mgr is y 2024-09-16T06:13:06.622 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T06:13:06.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-09-16T06:13:06.653 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-16T06:13:06.678 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T06:13:06.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:13:06.824 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56072/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-845-ge4e227b3', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-845.ge4e227b3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 17:59:03.516496', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T06:13:06.970 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 e4e227b37858f5109138494e01e5ba1ba7b8ffe8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:13:06.971 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm 2024-09-16T06:13:06.972 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T06:13:07.319 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 16 06:13 /home/ubuntu/cephtest/cephadm 2024-09-16T06:13:07.319 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T06:13:07.653 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 16 06:13 /home/ubuntu/cephtest/cephadm 2024-09-16T06:13:07.653 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T06:13:07.673 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T06:13:07.702 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 on all hosts... 2024-09-16T06:13:07.702 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 pull 2024-09-16T06:13:07.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 pull 2024-09-16T06:13:09.234 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T06:13:09.276 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T06:13:45.277 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-16T06:13:45.278 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T06:13:45.278 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1", 2024-09-16T06:13:45.278 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-16T06:13:45.278 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322" 2024-09-16T06:13:45.278 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-16T06:13:45.278 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-16T06:13:48.428 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-16T06:13:48.429 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T06:13:48.429 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1", 2024-09-16T06:13:48.429 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-09-16T06:13:48.429 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322" 2024-09-16T06:13:48.429 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-16T06:13:48.429 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-16T06:13:48.448 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-09-16T06:13:48.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-16T06:13:48.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-09-16T06:13:48.562 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-16T06:13:48.606 INFO:tasks.cephadm:Writing seed config... 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T06:13:48.607 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T06:13:48.608 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T06:13:48.608 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T06:13:48.608 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T06:13:48.608 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:13:48.608 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T06:13:48.624 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bd41205c-73f2-11ef-bceb-c7b262605968 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-16T06:13:48.624 DEBUG:teuthology.orchestra.run.smithi190:mon.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T06:13:48.667 DEBUG:teuthology.orchestra.run.smithi190:mgr.y> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T06:13:48.709 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T06:13:48.709 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 -v bootstrap --fsid bd41205c-73f2-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.190 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:13:48.862 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-16T06:13:48.862 INFO:teuthology.orchestra.run.smithi190.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8', '-v', 'bootstrap', '--fsid', 'bd41205c-73f2-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.190', '--skip-admin-label'] 2024-09-16T06:13:48.885 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stdout 5.2.2 2024-09-16T06:13:48.885 INFO:teuthology.orchestra.run.smithi190.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T06:13:48.886 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying podman|docker is present... 2024-09-16T06:13:48.916 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stdout 5.2.2 2024-09-16T06:13:48.917 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying lvm2 is present... 2024-09-16T06:13:48.917 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying time synchronization is in place... 2024-09-16T06:13:48.924 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T06:13:48.924 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T06:13:48.930 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T06:13:48.930 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout inactive 2024-09-16T06:13:48.936 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout enabled 2024-09-16T06:13:48.943 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout active 2024-09-16T06:13:48.943 INFO:teuthology.orchestra.run.smithi190.stdout:Unit chronyd.service is enabled and running 2024-09-16T06:13:48.943 INFO:teuthology.orchestra.run.smithi190.stdout:Repeating the final host check... 2024-09-16T06:13:48.970 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stdout 5.2.2 2024-09-16T06:13:48.970 INFO:teuthology.orchestra.run.smithi190.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T06:13:48.970 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl is present 2024-09-16T06:13:48.970 INFO:teuthology.orchestra.run.smithi190.stdout:lvcreate is present 2024-09-16T06:13:48.977 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T06:13:48.977 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T06:13:48.983 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T06:13:48.983 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout inactive 2024-09-16T06:13:48.989 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout enabled 2024-09-16T06:13:48.996 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout active 2024-09-16T06:13:48.996 INFO:teuthology.orchestra.run.smithi190.stdout:Unit chronyd.service is enabled and running 2024-09-16T06:13:48.996 INFO:teuthology.orchestra.run.smithi190.stdout:Host looks OK 2024-09-16T06:13:48.996 INFO:teuthology.orchestra.run.smithi190.stdout:Cluster fsid: bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:13:48.996 INFO:teuthology.orchestra.run.smithi190.stdout:Acquiring lock 139806683476128 on /run/cephadm/bd41205c-73f2-11ef-bceb-c7b262605968.lock 2024-09-16T06:13:48.996 INFO:teuthology.orchestra.run.smithi190.stdout:Lock 139806683476128 acquired on /run/cephadm/bd41205c-73f2-11ef-bceb-c7b262605968.lock 2024-09-16T06:13:48.997 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying IP 172.21.15.190 port 3300 ... 2024-09-16T06:13:48.997 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying IP 172.21.15.190 port 6789 ... 2024-09-16T06:13:48.997 INFO:teuthology.orchestra.run.smithi190.stdout:Base mon IP(s) is [172.21.15.190:3300, 172.21.15.190:6789], mon addrv is [v2:172.21.15.190:3300,v1:172.21.15.190:6789] 2024-09-16T06:13:49.000 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.190 metric 100 2024-09-16T06:13:49.000 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.190 metric 100 2024-09-16T06:13:49.002 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T06:13:49.003 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T06:13:49.003 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T06:13:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T06:13:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T06:13:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T06:13:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T06:13:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7763/64 scope link noprefixroute 2024-09-16T06:13:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T06:13:49.006 INFO:teuthology.orchestra.run.smithi190.stdout:Mon IP `172.21.15.190` is in CIDR network `172.21.0.0/20` 2024-09-16T06:13:49.006 INFO:teuthology.orchestra.run.smithi190.stdout:Mon IP `172.21.15.190` is in CIDR network `172.21.0.0/20` 2024-09-16T06:13:49.006 INFO:teuthology.orchestra.run.smithi190.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T06:13:49.006 INFO:teuthology.orchestra.run.smithi190.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T06:13:49.007 INFO:teuthology.orchestra.run.smithi190.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stdout 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stderr Copying blob sha256:71878612010b3dfa9e3e456d432a49e20ac26634831a8a6522f4d782181c5840 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stderr Copying config sha256:67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T06:13:49.449 INFO:teuthology.orchestra.run.smithi190.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T06:13:50.337 INFO:teuthology.orchestra.run.smithi190.stdout:ceph: stdout ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T06:13:50.337 INFO:teuthology.orchestra.run.smithi190.stdout:Ceph version: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T06:13:50.337 INFO:teuthology.orchestra.run.smithi190.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T06:13:51.191 INFO:teuthology.orchestra.run.smithi190.stdout:stat: stdout 167 167 2024-09-16T06:13:51.192 INFO:teuthology.orchestra.run.smithi190.stdout:Creating initial keys... 2024-09-16T06:13:56.360 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph-authtool: stdout AQCfzOdmat5WIBAA+1kf03XUQxwt8xQdk+46lQ== 2024-09-16T06:14:03.651 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph-authtool: stdout AQCnzOdmpMheFhAAUHeCZBq6ynKRGStBk94J4g== 2024-09-16T06:14:04.563 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph-authtool: stdout AQCrzOdmK5TCORAAcDHCvEpQDItVBOrpeHM9LQ== 2024-09-16T06:14:04.564 INFO:teuthology.orchestra.run.smithi190.stdout:Creating initial monmap... 2024-09-16T06:14:05.529 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:monmaptool for a [v2:172.21.15.190:3300,v1:172.21.15.190:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:setting min_mon_release = pacific 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/monmaptool: set fsid to bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:14:05.530 INFO:teuthology.orchestra.run.smithi190.stdout:Creating mon... 2024-09-16T06:14:06.543 INFO:teuthology.orchestra.run.smithi190.stdout:create mon.a on 2024-09-16T06:14:06.890 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T06:14:07.064 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bd41205c-73f2-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-bd41205c-73f2-11ef-bceb-c7b262605968.target. 2024-09-16T06:14:07.064 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bd41205c-73f2-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-bd41205c-73f2-11ef-bceb-c7b262605968.target. 2024-09-16T06:14:07.292 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a 2024-09-16T06:14:07.293 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T06:14:07.452 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bd41205c-73f2-11ef-bceb-c7b262605968.target.wants/ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bd41205c-73f2-11ef-bceb-c7b262605968@.service. 2024-09-16T06:14:07.942 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:07 smithi190 podman[25380]: 2024-09-16 06:14:07.899827636 +0000 UTC m=+0.324946830 container start e8779dcfe4b8d52bfcb0aee735e07fc1e90444352c25cf969f16020597587e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T06:14:08.000 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T06:14:08.000 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout disabled 2024-09-16T06:14:08.007 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T06:14:08.007 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout inactive 2024-09-16T06:14:08.007 INFO:teuthology.orchestra.run.smithi190.stdout:firewalld.service is not enabled 2024-09-16T06:14:08.007 INFO:teuthology.orchestra.run.smithi190.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T06:14:08.008 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for mon to start... 2024-09-16T06:14:08.008 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for mon... 2024-09-16T06:14:08.242 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:07 smithi190 ceph-mon[25418]: mkfs bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:08.243 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:07 smithi190 ceph-mon[25418]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T06:14:08.243 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:07 smithi190 bash[25380]: e8779dcfe4b8d52bfcb0aee735e07fc1e90444352c25cf969f16020597587e9b 2024-09-16T06:14:08.243 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:07 smithi190 systemd[1]: Started Ceph mon.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:14:09.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:08 smithi190 ceph-mon[25418]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T06:14:09.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:08 smithi190 ceph-mon[25418]: monmap e1: 1 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:14:09.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:08 smithi190 ceph-mon[25418]: fsmap 2024-09-16T06:14:09.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:08 smithi190 ceph-mon[25418]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T06:14:09.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:08 smithi190 ceph-mon[25418]: mgrmap e1: no daemons active 2024-09-16T06:14:09.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:08 smithi190 ceph-mon[25418]: from='client.? 172.21.15.190:0/3821492306' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout id: bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout services: 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.572452s) 2024-09-16T06:14:10.319 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout data: 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:mon is available 2024-09-16T06:14:10.320 INFO:teuthology.orchestra.run.smithi190.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T06:14:11.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:10 smithi190 ceph-mon[25418]: from='client.? 172.21.15.190:0/1938312868' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T06:14:11.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:10 smithi190 ceph-mon[25418]: from='client.? 172.21.15.190:0/1938312868' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout fsid = bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.190:3300,v1:172.21.15.190:6789] 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T06:14:11.459 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-09-16T06:14:11.460 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T06:14:11.460 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:11.460 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T06:14:11.460 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T06:14:11.460 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T06:14:11.460 INFO:teuthology.orchestra.run.smithi190.stdout:Generating new minimal ceph.conf... 2024-09-16T06:14:12.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:11 smithi190 ceph-mon[25418]: from='client.? 172.21.15.190:0/368453388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:14:12.497 INFO:teuthology.orchestra.run.smithi190.stdout:Restarting the monitor... 2024-09-16T06:14:12.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:12 smithi190 systemd[1]: Stopping Ceph mon.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:14:12.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:12 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a[25395]: 2024-09-16T06:14:12.765+0000 7fc5de9fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:14:12.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:12 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a[25395]: 2024-09-16T06:14:12.765+0000 7fc5de9fa640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T06:14:13.180 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:12 smithi190 podman[25736]: 2024-09-16 06:14:12.919005989 +0000 UTC m=+0.255256191 container died e8779dcfe4b8d52bfcb0aee735e07fc1e90444352c25cf969f16020597587e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b) 2024-09-16T06:14:13.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 podman[25736]: 2024-09-16 06:14:13.293751661 +0000 UTC m=+0.630001864 container cleanup e8779dcfe4b8d52bfcb0aee735e07fc1e90444352c25cf969f16020597587e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T06:14:13.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 bash[25736]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a 2024-09-16T06:14:13.973 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 podman[25753]: 2024-09-16 06:14:13.59781641 +0000 UTC m=+0.675022188 container remove e8779dcfe4b8d52bfcb0aee735e07fc1e90444352c25cf969f16020597587e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T06:14:13.973 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T06:14:13.973 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 systemd[1]: Stopped Ceph mon.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:14:13.973 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 systemd[1]: Starting Ceph mon.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:14:14.251 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 podman[25850]: 2024-09-16 06:14:13.875815731 +0000 UTC m=+0.015582554 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:14:14.251 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:13 smithi190 podman[25850]: 2024-09-16 06:14:13.98204552 +0000 UTC m=+0.121812335 container create af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:14:14.251 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 podman[25850]: 2024-09-16 06:14:14.15940993 +0000 UTC m=+0.299176750 container init af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b) 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 podman[25850]: 2024-09-16 06:14:14.164204413 +0000 UTC m=+0.303971221 container start af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 7 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: pidfile_write: ignore empty --pid-file 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: load: jerasure load: lrc 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: RocksDB version: 7.9.2 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Git sha 0 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: DB SUMMARY 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: DB Session ID: DOQR8LEM4S4LQZJF707K 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: CURRENT file: CURRENT 2024-09-16T06:14:14.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83762 ; 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.error_if_exists: 0 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.create_if_missing: 0 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.paranoid_checks: 1 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.env: 0x5652a7e79bc0 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.info_log: 0x5652a86dcbe0 2024-09-16T06:14:14.253 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.statistics: (nil) 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.use_fsync: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_log_file_size: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.allow_fallocate: 1 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.use_direct_reads: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T06:14:14.254 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.db_log_dir: 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.wal_dir: 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.write_buffer_manager: 0x5652a86ea320 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T06:14:14.255 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T06:14:14.256 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T06:14:14.256 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T06:14:14.256 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.unordered_write: 0 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T06:14:14.257 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.row_cache: None 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.wal_filter: None 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.two_write_queues: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.wal_compression: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.atomic_flush: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.log_readahead_size: 0 2024-09-16T06:14:14.258 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_background_jobs: 2 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_background_compactions: -1 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_subcompactions: 1 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T06:14:14.259 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_open_files: -1 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_background_flushes: -1 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Compression algorithms supported: 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kZSTD supported: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kXpressCompression supported: 0 2024-09-16T06:14:14.260 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kBZip2Compression supported: 0 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kLZ4Compression supported: 1 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kZlibCompression supported: 1 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: kSnappyCompression supported: 1 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.merge_operator: 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_filter: None 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T06:14:14.261 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5652a86dc7e0) 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: cache_index_and_filter_blocks: 1 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: pin_top_level_index_and_filter: 1 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: index_type: 0 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: data_block_index_type: 0 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: index_shortening: 1 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T06:14:14.262 INFO:journalctl@ceph.mon.a.smithi190.stdout: checksum: 4 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: no_block_cache: 0 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_cache: 0x5652a86d1350 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_cache_name: BinnedLRUCache 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_cache_options: 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: capacity : 536870912 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: num_shard_bits : 4 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: strict_capacity_limit : 0 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: high_pri_pool_ratio: 0.000 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_cache_compressed: (nil) 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: persistent_cache: (nil) 2024-09-16T06:14:14.263 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_size: 4096 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_size_deviation: 10 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_restart_interval: 16 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: index_block_restart_interval: 1 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: metadata_block_size: 4096 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: partition_filters: 0 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: use_delta_encoding: 1 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: filter_policy: bloomfilter 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: whole_key_filtering: 1 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: verify_compression: 0 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: read_amp_bytes_per_bit: 0 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: format_version: 5 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: enable_index_compression: 1 2024-09-16T06:14:14.264 INFO:journalctl@ceph.mon.a.smithi190.stdout: block_align: 0 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout: max_auto_readahead_size: 262144 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout: prepopulate_block_cache: 0 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout: initial_auto_readahead_size: 8192 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression: NoCompression 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.num_levels: 7 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T06:14:14.265 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T06:14:14.266 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T06:14:14.267 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T06:14:14.268 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T06:14:14.269 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.inplace_update_support: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.bloom_locality: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.max_successive_merges: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.ttl: 2592000 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T06:14:14.270 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.enable_blob_files: false 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.min_blob_size: 0 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T06:14:14.271 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 63f5585b-4c74-4af8-a47d-4449af160ae8 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467254190790, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467254191664, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80535, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78696, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 73004, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726467254, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "63f5585b-4c74-4af8-a47d-4449af160ae8", "db_session_id": "DOQR8LEM4S4LQZJF707K", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467254191761, "job": 1, "event": "recovery_finished"} 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5652a86fce00 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: DB pointer 0x5652a87f0000 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout: ** DB Stats ** 2024-09-16T06:14:14.272 INFO:journalctl@ceph.mon.a.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: ** Compaction Stats [default] ** 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: L0 2/0 80.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Sum 2/0 80.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T06:14:14.273 INFO:journalctl@ceph.mon.a.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: ** Compaction Stats [default] ** 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T06:14:14.274 INFO:journalctl@ceph.mon.a.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: Cumulative compaction: 0.00 GB write, 25.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: Interval compaction: 0.00 GB write, 25.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: Block cache BinnedLRUCache@0x5652a86d1350#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: starting mon.a rank 0 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???) e1 preinit fsid bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).mds e1 new map 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).mds e1 print_map 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: e1 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T06:14:14.275 INFO:journalctl@ceph.mon.a.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout: legacy client fscid: -1 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout: 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout: No filesystems configured 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 bash[25850]: af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 2024-09-16T06:14:14.276 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 systemd[1]: Started Ceph mon.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:14:14.276 INFO:teuthology.orchestra.run.smithi190.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T06:14:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T06:14:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: monmap e1: 1 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:14:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: fsmap 2024-09-16T06:14:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T06:14:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:14 smithi190 ceph-mon[25888]: mgrmap e1: no daemons active 2024-09-16T06:14:15.727 INFO:teuthology.orchestra.run.smithi190.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T06:14:15.748 INFO:teuthology.orchestra.run.smithi190.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:14:15.748 INFO:teuthology.orchestra.run.smithi190.stdout:Creating mgr... 2024-09-16T06:14:15.749 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T06:14:15.750 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T06:14:15.991 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y 2024-09-16T06:14:15.991 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-16T06:14:16.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:15 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3711739844' entity='client.admin' 2024-09-16T06:14:16.141 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bd41205c-73f2-11ef-bceb-c7b262605968.target.wants/ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-bd41205c-73f2-11ef-bceb-c7b262605968@.service. 2024-09-16T06:14:16.464 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 podman[26127]: 2024-09-16 06:14:16.275929311 +0000 UTC m=+0.016096852 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:14:16.645 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T06:14:16.645 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout disabled 2024-09-16T06:14:16.654 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T06:14:16.654 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout inactive 2024-09-16T06:14:16.654 INFO:teuthology.orchestra.run.smithi190.stdout:firewalld.service is not enabled 2024-09-16T06:14:16.654 INFO:teuthology.orchestra.run.smithi190.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T06:14:16.664 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T06:14:16.664 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout disabled 2024-09-16T06:14:16.673 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T06:14:16.673 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout inactive 2024-09-16T06:14:16.673 INFO:teuthology.orchestra.run.smithi190.stdout:firewalld.service is not enabled 2024-09-16T06:14:16.673 INFO:teuthology.orchestra.run.smithi190.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T06:14:16.673 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for mgr to start... 2024-09-16T06:14:16.673 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for mgr... 2024-09-16T06:14:16.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 podman[26127]: 2024-09-16 06:14:16.553502629 +0000 UTC m=+0.293670156 container init 057fa89f58e3102a32969b98536b3511a9fa8d25243e1282161f3c3904bf5a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T06:14:16.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 podman[26127]: 2024-09-16 06:14:16.558320554 +0000 UTC m=+0.298488084 container start 057fa89f58e3102a32969b98536b3511a9fa8d25243e1282161f3c3904bf5a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T06:14:16.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 bash[26127]: 057fa89f58e3102a32969b98536b3511a9fa8d25243e1282161f3c3904bf5a43 2024-09-16T06:14:16.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 systemd[1]: Started Ceph mgr.y for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:14:16.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:16.685+0000 7ff1b89f8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:14:16.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:16.743+0000 7ff1b89f8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:14:17.189 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:16.898+0000 7ff1b89f8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:14:17.190 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:16 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:16.967+0000 7ff1b89f8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:14:17.190 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:17.114+0000 7ff1b89f8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:14:17.484 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:17 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3759396710' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T06:14:17.484 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:17.269+0000 7ff1b89f8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:14:17.776 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:17.586+0000 7ff1b89f8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:14:17.776 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:17.647+0000 7ff1b89f8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:14:17.781 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "fsid": "bd41205c-73f2-11ef-bceb-c7b262605968", 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T06:14:17.782 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 0 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T06:14:17.783 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:17.784 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T06:14:17.784 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T06:14:17.784 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T06:14:17.784 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T06:14:17.787 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T06:14:17.787 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.788 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T06:14:17.789 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T06:14:07.938974+0000", 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:17.790 INFO:teuthology.orchestra.run.smithi190.stdout:mgr not available, waiting (1/15)... 2024-09-16T06:14:18.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:17.832+0000 7ff1b89f8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:14:19.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:18 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:18.692+0000 7ff1b89f8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:14:19.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:18 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:18.757+0000 7ff1b89f8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:14:19.773 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:19 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:19.464+0000 7ff1b89f8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T06:14:19.773 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:19 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:19.576+0000 7ff1b89f8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:14:19.773 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:19 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:19.713+0000 7ff1b89f8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:14:19.774 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:19 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:19.772+0000 7ff1b89f8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:14:20.028 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:19 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:19.940+0000 7ff1b89f8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T06:14:20.029 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.011+0000 7ff1b89f8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:14:20.311 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.073+0000 7ff1b89f8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:14:20.311 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.242+0000 7ff1b89f8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:14:20.311 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.310+0000 7ff1b89f8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:14:20.584 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.380+0000 7ff1b89f8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:14:20.584 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.481+0000 7ff1b89f8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:14:20.584 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.547+0000 7ff1b89f8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:14:20.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2348291300' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T06:14:20.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:20 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:20.613+0000 7ff1b89f8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "fsid": "bd41205c-73f2-11ef-bceb-c7b262605968", 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T06:14:21.097 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 0 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T06:14:21.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:21.099 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T06:14:21.099 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T06:14:21.099 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T06:14:21.099 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T06:14:21.102 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T06:14:21.102 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T06:14:21.102 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T06:14:21.103 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:21.104 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T06:14:07.938974+0000", 2024-09-16T06:14:21.105 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T06:14:21.105 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:21.105 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T06:14:21.105 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:21.105 INFO:teuthology.orchestra.run.smithi190.stdout:mgr not available, waiting (2/15)... 2024-09-16T06:14:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:21.404+0000 7ff1b89f8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:14:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: Activating manager daemon y 2024-09-16T06:14:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: mgrmap e2: y(active, starting, since 0.00296489s) 2024-09-16T06:14:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T06:14:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: Manager daemon y is now available 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' 2024-09-16T06:14:21.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:21 smithi190 ceph-mon[25888]: from='mgr.14100 172.21.15.190:0/3117366415' entity='mgr.y' 2024-09-16T06:14:23.544 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:23 smithi190 ceph-mon[25888]: mgrmap e3: y(active, since 1.00688s) 2024-09-16T06:14:24.381 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:24.381 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "fsid": "bd41205c-73f2-11ef-bceb-c7b262605968", 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 0 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T06:14:24.382 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T06:14:24.383 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.384 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T06:14:24.385 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ], 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T06:14:07.938974+0000", 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout }, 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:24.386 INFO:teuthology.orchestra.run.smithi190.stdout:mgr is available 2024-09-16T06:14:24.632 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:24 smithi190 ceph-mon[25888]: mgrmap e4: y(active, since 2s) 2024-09-16T06:14:24.632 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:24 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/434602692' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T06:14:25.588 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:25 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2419618362' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T06:14:25.588 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:25 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2419618362' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout fsid = bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 2024-09-16T06:14:25.592 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T06:14:25.593 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T06:14:25.593 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T06:14:25.593 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling cephadm module... 2024-09-16T06:14:26.761 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/86165173' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T06:14:26.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ignoring --setuser ceph since I am not root 2024-09-16T06:14:26.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ignoring --setgroup ceph since I am not root 2024-09-16T06:14:26.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:26.557+0000 7f1274ed6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:14:26.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:26.613+0000 7f1274ed6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:14:26.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:26.760+0000 7f1274ed6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:14:27.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:26.826+0000 7f1274ed6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:14:27.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:26.969+0000 7f1274ed6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:14:27.391 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:27 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:27.114+0000 7f1274ed6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:14:27.673 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:27 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/86165173' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T06:14:27.673 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:27 smithi190 ceph-mon[25888]: mgrmap e5: y(active, since 5s) 2024-09-16T06:14:27.673 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:27 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:27.419+0000 7f1274ed6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:14:27.674 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:27 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:27.481+0000 7f1274ed6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:14:27.674 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:27 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:27.672+0000 7f1274ed6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:14:28.453 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for the mgr to restart... 2024-09-16T06:14:28.454 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for mgr epoch 5... 2024-09-16T06:14:28.677 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:28 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/81701999' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T06:14:28.678 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:28 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:28.552+0000 7f1274ed6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:14:28.678 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:28 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:28.617+0000 7f1274ed6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:14:29.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.328+0000 7f1274ed6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T06:14:29.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.439+0000 7f1274ed6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:14:29.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.578+0000 7f1274ed6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:14:29.905 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.637+0000 7f1274ed6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:14:29.905 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.780+0000 7f1274ed6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T06:14:29.906 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.846+0000 7f1274ed6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:14:29.906 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:29.905+0000 7f1274ed6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:14:30.185 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:30.057+0000 7f1274ed6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:14:30.185 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:30.120+0000 7f1274ed6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:14:30.185 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:30.184+0000 7f1274ed6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:14:30.491 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:30.265+0000 7f1274ed6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:14:30.491 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:30.323+0000 7f1274ed6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:14:30.491 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:30.379+0000 7f1274ed6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:31.150+0000 7f1274ed6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: Active manager daemon y restarted 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: Activating manager daemon y 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: mgrmap e6: y(active, starting, since 0.00376757s) 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T06:14:31.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: Manager daemon y is now available 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T06:14:31.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:31 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T06:14:32.456 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:32 smithi190 ceph-mon[25888]: Found migration_current of "None". Setting to last migration. 2024-09-16T06:14:32.456 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:32 smithi190 ceph-mon[25888]: mgrmap e7: y(active, since 1.00575s) 2024-09-16T06:14:32.717 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:32.718 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T06:14:32.718 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T06:14:32.718 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:32.718 INFO:teuthology.orchestra.run.smithi190.stdout:mgr epoch 5 is available 2024-09-16T06:14:32.718 INFO:teuthology.orchestra.run.smithi190.stdout:Setting orchestrator backend to cephadm... 2024-09-16T06:14:33.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-mon[25888]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T06:14:33.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-mon[25888]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T06:14:33.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:33.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:33.952+0000 7f11dcbb6640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:06:14:33] ENGINE Error in HTTPServer.serve 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: Traceback (most recent call last): 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._connections.run(self.expiration_interval) 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._run(expiration_interval) 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:14:34.052 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: s = self.context.wrap_socket( 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: return self.sslsocket_class._create( 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self.do_handshake() 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._sslobj.do_handshake() 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:14:34.053 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:33 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:34.558 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:34 smithi190 ceph-mon[25888]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:34.559 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:34 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:34.559 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:34 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:14:34.559 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:34 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:14:34.559 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:34 smithi190 ceph-mon[25888]: mgrmap e8: y(active, since 2s) 2024-09-16T06:14:35.137 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T06:14:35.138 INFO:teuthology.orchestra.run.smithi190.stdout:Generating ssh key... 2024-09-16T06:14:35.397 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:35 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:33] ENGINE Bus STARTING 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:35 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:33] ENGINE Serving on http://172.21.15.190:8765 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:35 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:33] ENGINE Serving on https://172.21.15.190:7150 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:35 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:33] ENGINE Bus STARTED 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:35 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:33] ENGINE Error in HTTPServer.serve 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: Traceback (most recent call last): 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._connections.run(self.expiration_interval) 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._run(expiration_interval) 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:14:35.398 INFO:journalctl@ceph.mon.a.smithi190.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: s = self.context.wrap_socket( 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: return self.sslsocket_class._create( 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: self.do_handshake() 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._sslobj.do_handshake() 2024-09-16T06:14:35.399 INFO:journalctl@ceph.mon.a.smithi190.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-mon[25888]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: Generating public/private rsa key pair. 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: Your identification has been saved in /tmp/tmpsh20_czc/key 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: Your public key has been saved in /tmp/tmpsh20_czc/key.pub 2024-09-16T06:14:36.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: The key fingerprint is: 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: SHA256:8SMckKbswErCyMKFeY3f2BUAGw2nhqLeSGzwaGRudWI ceph-bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: The key's randomart image is: 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: +---[RSA 3072]----+ 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: | o o+=+... | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: | o +..Bo . | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: |*=oE.B+ + | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: |@B* *o + + | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: |*O.o S o | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: |B o . . . | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: | o . | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: | | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: | | 2024-09-16T06:14:36.598 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: +----[SHA256]-----+ 2024-09-16T06:14:37.689 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:37 smithi190 ceph-mon[25888]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:37.689 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:37 smithi190 ceph-mon[25888]: Generating ssh key... 2024-09-16T06:14:37.941 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3iGlUeMg5FCxZT/rjDGfSjOiHwZBqWmdRg3WKxmr5XIp4uciMWy0YXQcq48QsV368G45vqUCDq1G2yVJOG/Ji75fAcJz3rnvzcSnPvDGOw55OZPJhpeEbeeG4+xy1A1SUQnSVTFlUX+QnH8QBMh57zMPAYjGKlma7kBsrji7YABWL4Nu9WfSKSwS5a+5Bv0ojd8k9VTJX79r3lVLq31LE7pkh6UnXxDOOGHYxIWQ8TzPEXPaAaYGVBHBmi0leWX+L0/6dSaSGjY7btDYNKYazLHq1UrobQ3csgQStDRGQRL9Vk8zvTSDdjKwUWHX43sqKb9I+etD5nKQUA1yxpDQeS21chBInxprGbGUnhLFlOiCdZUw1B7OYtLIftWB6TV+N0+RAMqh0mKqyrwIfRCidsXSUUkRIwGsaWhuP7pP3bfD/M7FTO7C8FvhV5dZGFKQWNQJ3eLb2A0eX8jc9jLnrhMS/383RLy1LYvSsMXem8VJH8ArFcHCrOZL3wEw6Ya0= ceph-bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:14:37.942 INFO:teuthology.orchestra.run.smithi190.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T06:14:37.942 INFO:teuthology.orchestra.run.smithi190.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T06:14:37.949 INFO:teuthology.orchestra.run.smithi190.stdout:Adding host smithi190... 2024-09-16T06:14:38.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:38 smithi190 ceph-mon[25888]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:39.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:39 smithi190 ceph-mon[25888]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "addr": "172.21.15.190", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:39.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:39 smithi190 ceph-mon[25888]: Deploying cephadm binary to smithi190 2024-09-16T06:14:41.808 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout Added host 'smithi190' with addr '172.21.15.190' 2024-09-16T06:14:41.808 INFO:teuthology.orchestra.run.smithi190.stdout:Deploying unmanaged mon service... 2024-09-16T06:14:42.357 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:42 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:42.358 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:42 smithi190 ceph-mon[25888]: Added host smithi190 2024-09-16T06:14:42.358 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:42 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:14:43.810 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:43 smithi190 ceph-mon[25888]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:43.810 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:43 smithi190 ceph-mon[25888]: Saving service mon spec with placement count:5 2024-09-16T06:14:43.810 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:43 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:43.810 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:43 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:43.812 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T06:14:43.812 INFO:teuthology.orchestra.run.smithi190.stdout:Deploying unmanaged mgr service... 2024-09-16T06:14:45.167 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:44 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:45.167 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:44 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:45.170 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T06:14:46.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:45 smithi190 ceph-mon[25888]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:46.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:45 smithi190 ceph-mon[25888]: Saving service mgr spec with placement count:2 2024-09-16T06:14:46.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:45 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3593283419' entity='client.admin' 2024-09-16T06:14:47.402 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:47 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:47.402 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:47 smithi190 ceph-mon[25888]: from='mgr.14118 172.21.15.190:0/1618805410' entity='mgr.y' 2024-09-16T06:14:48.005 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling the dashboard module... 2024-09-16T06:14:48.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:48 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/519094884' entity='client.admin' 2024-09-16T06:14:49.687 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2581164582' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T06:14:49.687 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ignoring --setuser ceph since I am not root 2024-09-16T06:14:49.687 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ignoring --setgroup ceph since I am not root 2024-09-16T06:14:49.687 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:49.545+0000 7f8775b71040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:14:49.688 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:49.601+0000 7f8775b71040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:14:49.946 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:49.752+0000 7f8775b71040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:14:49.946 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:49.818+0000 7f8775b71040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:14:50.220 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:49 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:49.972+0000 7f8775b71040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:14:50.221 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:50 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:50.114+0000 7f8775b71040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:14:50.474 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:50 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2581164582' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T06:14:50.474 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:50 smithi190 ceph-mon[25888]: mgrmap e9: y(active, since 18s) 2024-09-16T06:14:50.474 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:50 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:50.435+0000 7f8775b71040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:14:50.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:50 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:50.503+0000 7f8775b71040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:14:50.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:50 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:50.695+0000 7f8775b71040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:14:51.255 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:51.255 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T06:14:51.255 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T06:14:51.255 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T06:14:51.255 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T06:14:51.256 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:51.256 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for the mgr to restart... 2024-09-16T06:14:51.256 INFO:teuthology.orchestra.run.smithi190.stdout:Waiting for mgr epoch 9... 2024-09-16T06:14:51.508 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:51 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1548632589' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T06:14:51.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:51 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:51.637+0000 7f8775b71040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:14:51.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:51 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:51.707+0000 7f8775b71040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:14:52.672 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:52.418+0000 7f8775b71040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T06:14:52.672 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:52.533+0000 7f8775b71040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:14:52.945 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:52.671+0000 7f8775b71040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:14:52.945 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:52.732+0000 7f8775b71040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:14:52.945 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:52.877+0000 7f8775b71040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T06:14:52.945 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:52.944+0000 7f8775b71040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:14:53.222 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.003+0000 7f8775b71040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:14:53.222 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.157+0000 7f8775b71040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:14:53.222 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.221+0000 7f8775b71040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:14:53.484 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.286+0000 7f8775b71040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:14:53.485 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.367+0000 7f8775b71040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:14:53.485 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.425+0000 7f8775b71040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:14:53.485 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:53.484+0000 7f8775b71040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:14:54.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:54.261+0000 7f8775b71040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:14:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: Active manager daemon y restarted 2024-09-16T06:14:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: Activating manager daemon y 2024-09-16T06:14:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T06:14:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: mgrmap e10: y(active, starting, since 0.00331019s) 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: Manager daemon y is now available 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T06:14:54.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T06:14:55.719 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:55.584+0000 7f86e1806640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:06:14:55] ENGINE Error in HTTPServer.serve 2024-09-16T06:14:55.719 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: Traceback (most recent call last): 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._connections.run(self.expiration_interval) 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._run(expiration_interval) 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: s = self.context.wrap_socket( 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: return self.sslsocket_class._create( 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:14:55.720 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self.do_handshake() 2024-09-16T06:14:55.721 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:14:55.721 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._sslobj.do_handshake() 2024-09-16T06:14:55.721 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:14:55.721 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:14:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:14:55.996 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout { 2024-09-16T06:14:55.997 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T06:14:55.997 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T06:14:55.997 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout } 2024-09-16T06:14:55.997 INFO:teuthology.orchestra.run.smithi190.stdout:mgr epoch 9 is available 2024-09-16T06:14:55.997 INFO:teuthology.orchestra.run.smithi190.stdout:Generating a dashboard self-signed certificate... 2024-09-16T06:14:56.564 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:56 smithi190 ceph-mon[25888]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T06:14:56.564 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:56 smithi190 ceph-mon[25888]: mgrmap e11: y(active, since 1.00568s) 2024-09-16T06:14:56.564 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:56 smithi190 ceph-mon[25888]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T06:14:56.564 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:56 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:14:57.569 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:55] ENGINE Bus STARTING 2024-09-16T06:14:57.569 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:55] ENGINE Serving on https://172.21.15.190:7150 2024-09-16T06:14:57.569 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:55] ENGINE Error in HTTPServer.serve 2024-09-16T06:14:57.569 INFO:journalctl@ceph.mon.a.smithi190.stdout: Traceback (most recent call last): 2024-09-16T06:14:57.569 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._connections.run(self.expiration_interval) 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._run(expiration_interval) 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: s = self.context.wrap_socket( 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: return self.sslsocket_class._create( 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:14:57.570 INFO:journalctl@ceph.mon.a.smithi190.stdout: self.do_handshake() 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._sslobj.do_handshake() 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:55] ENGINE Serving on http://172.21.15.190:8765 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: [16/Sep/2024:06:14:55] ENGINE Bus STARTED 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:14:57.571 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:57 smithi190 ceph-mon[25888]: mgrmap e12: y(active, since 2s) 2024-09-16T06:14:57.572 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T06:14:57.572 INFO:teuthology.orchestra.run.smithi190.stdout:Creating initial admin user... 2024-09-16T06:14:58.346 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:58 smithi190 ceph-mon[25888]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:14:59.490 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$uCjOKXjPPYe1byAY50fKkORLDbU68DSf7B2q6mZNVvSmDd1lIPnvK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726467298, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T06:14:59.490 INFO:teuthology.orchestra.run.smithi190.stdout:Fetching dashboard port number... 2024-09-16T06:15:00.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:59 smithi190 ceph-mon[25888]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:00.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:14:59 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:00.653 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T06:15:00.663 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T06:15:00.663 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout disabled 2024-09-16T06:15:00.670 INFO:teuthology.orchestra.run.smithi190.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T06:15:00.670 INFO:teuthology.orchestra.run.smithi190.stdout:systemctl: stdout inactive 2024-09-16T06:15:00.670 INFO:teuthology.orchestra.run.smithi190.stdout:firewalld.service is not enabled 2024-09-16T06:15:00.670 INFO:teuthology.orchestra.run.smithi190.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T06:15:00.671 INFO:teuthology.orchestra.run.smithi190.stdout:Ceph Dashboard is now available at: 2024-09-16T06:15:00.671 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:00.671 INFO:teuthology.orchestra.run.smithi190.stdout: URL: https://smithi190.front.sepia.ceph.com:8443/ 2024-09-16T06:15:00.671 INFO:teuthology.orchestra.run.smithi190.stdout: User: admin 2024-09-16T06:15:00.671 INFO:teuthology.orchestra.run.smithi190.stdout: Password: 31ldf17xz3 2024-09-16T06:15:00.672 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:00.672 INFO:teuthology.orchestra.run.smithi190.stdout:Saving cluster configuration to /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config directory 2024-09-16T06:15:00.672 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling autotune for osd_memory_target 2024-09-16T06:15:01.315 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:00 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3178969654' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T06:15:02.210 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:01 smithi190 ceph-mon[25888]: mgrmap e13: y(active, since 6s) 2024-09-16T06:15:03.098 INFO:teuthology.orchestra.run.smithi190.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: ceph telemetry on 2024-09-16T06:15:03.099 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.100 INFO:teuthology.orchestra.run.smithi190.stdout:For more information see: 2024-09-16T06:15:03.100 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.100 INFO:teuthology.orchestra.run.smithi190.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T06:15:03.100 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:03.100 INFO:teuthology.orchestra.run.smithi190.stdout:Bootstrap complete. 2024-09-16T06:15:03.132 INFO:tasks.cephadm:Fetching config... 2024-09-16T06:15:03.132 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:03.132 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T06:15:03.148 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T06:15:03.148 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:03.149 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T06:15:03.203 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T06:15:03.204 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:03.204 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T06:15:03.268 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T06:15:03.269 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:03.269 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T06:15:03.324 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T06:15:03.324 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3iGlUeMg5FCxZT/rjDGfSjOiHwZBqWmdRg3WKxmr5XIp4uciMWy0YXQcq48QsV368G45vqUCDq1G2yVJOG/Ji75fAcJz3rnvzcSnPvDGOw55OZPJhpeEbeeG4+xy1A1SUQnSVTFlUX+QnH8QBMh57zMPAYjGKlma7kBsrji7YABWL4Nu9WfSKSwS5a+5Bv0ojd8k9VTJX79r3lVLq31LE7pkh6UnXxDOOGHYxIWQ8TzPEXPaAaYGVBHBmi0leWX+L0/6dSaSGjY7btDYNKYazLHq1UrobQ3csgQStDRGQRL9Vk8zvTSDdjKwUWHX43sqKb9I+etD5nKQUA1yxpDQeS21chBInxprGbGUnhLFlOiCdZUw1B7OYtLIftWB6TV+N0+RAMqh0mKqyrwIfRCidsXSUUkRIwGsaWhuP7pP3bfD/M7FTO7C8FvhV5dZGFKQWNQJ3eLb2A0eX8jc9jLnrhMS/383RLy1LYvSsMXem8VJH8ArFcHCrOZL3wEw6Ya0= ceph-bd41205c-73f2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T06:15:03.426 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3iGlUeMg5FCxZT/rjDGfSjOiHwZBqWmdRg3WKxmr5XIp4uciMWy0YXQcq48QsV368G45vqUCDq1G2yVJOG/Ji75fAcJz3rnvzcSnPvDGOw55OZPJhpeEbeeG4+xy1A1SUQnSVTFlUX+QnH8QBMh57zMPAYjGKlma7kBsrji7YABWL4Nu9WfSKSwS5a+5Bv0ojd8k9VTJX79r3lVLq31LE7pkh6UnXxDOOGHYxIWQ8TzPEXPaAaYGVBHBmi0leWX+L0/6dSaSGjY7btDYNKYazLHq1UrobQ3csgQStDRGQRL9Vk8zvTSDdjKwUWHX43sqKb9I+etD5nKQUA1yxpDQeS21chBInxprGbGUnhLFlOiCdZUw1B7OYtLIftWB6TV+N0+RAMqh0mKqyrwIfRCidsXSUUkRIwGsaWhuP7pP3bfD/M7FTO7C8FvhV5dZGFKQWNQJ3eLb2A0eX8jc9jLnrhMS/383RLy1LYvSsMXem8VJH8ArFcHCrOZL3wEw6Ya0= ceph-bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:03.453 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3iGlUeMg5FCxZT/rjDGfSjOiHwZBqWmdRg3WKxmr5XIp4uciMWy0YXQcq48QsV368G45vqUCDq1G2yVJOG/Ji75fAcJz3rnvzcSnPvDGOw55OZPJhpeEbeeG4+xy1A1SUQnSVTFlUX+QnH8QBMh57zMPAYjGKlma7kBsrji7YABWL4Nu9WfSKSwS5a+5Bv0ojd8k9VTJX79r3lVLq31LE7pkh6UnXxDOOGHYxIWQ8TzPEXPaAaYGVBHBmi0leWX+L0/6dSaSGjY7btDYNKYazLHq1UrobQ3csgQStDRGQRL9Vk8zvTSDdjKwUWHX43sqKb9I+etD5nKQUA1yxpDQeS21chBInxprGbGUnhLFlOiCdZUw1B7OYtLIftWB6TV+N0+RAMqh0mKqyrwIfRCidsXSUUkRIwGsaWhuP7pP3bfD/M7FTO7C8FvhV5dZGFKQWNQJ3eLb2A0eX8jc9jLnrhMS/383RLy1LYvSsMXem8VJH8ArFcHCrOZL3wEw6Ya0= ceph-bd41205c-73f2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T06:15:03.502 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3iGlUeMg5FCxZT/rjDGfSjOiHwZBqWmdRg3WKxmr5XIp4uciMWy0YXQcq48QsV368G45vqUCDq1G2yVJOG/Ji75fAcJz3rnvzcSnPvDGOw55OZPJhpeEbeeG4+xy1A1SUQnSVTFlUX+QnH8QBMh57zMPAYjGKlma7kBsrji7YABWL4Nu9WfSKSwS5a+5Bv0ojd8k9VTJX79r3lVLq31LE7pkh6UnXxDOOGHYxIWQ8TzPEXPaAaYGVBHBmi0leWX+L0/6dSaSGjY7btDYNKYazLHq1UrobQ3csgQStDRGQRL9Vk8zvTSDdjKwUWHX43sqKb9I+etD5nKQUA1yxpDQeS21chBInxprGbGUnhLFlOiCdZUw1B7OYtLIftWB6TV+N0+RAMqh0mKqyrwIfRCidsXSUUkRIwGsaWhuP7pP3bfD/M7FTO7C8FvhV5dZGFKQWNQJ3eLb2A0eX8jc9jLnrhMS/383RLy1LYvSsMXem8VJH8ArFcHCrOZL3wEw6Ya0= ceph-bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:03.515 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T06:15:03.674 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:03 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3180953351' entity='client.admin' 2024-09-16T06:15:03.741 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:06.589 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T06:15:06.589 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T06:15:06.860 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:07.230 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:06 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/608291307' entity='client.admin' 2024-09-16T06:15:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:15:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:09.992 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-16T06:15:09.992 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:15:09.992 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-16T06:15:10.012 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:15:10.013 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:10.071 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-16T06:15:10.071 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch host add smithi196 2024-09-16T06:15:10.260 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T06:15:10.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:15:10.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:10.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:10.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:11.410 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:11 smithi190 ceph-mon[25888]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:11.410 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:11 smithi190 ceph-mon[25888]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:15:12.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:12 smithi190 ceph-mon[25888]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:13.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:13 smithi190 ceph-mon[25888]: Deploying cephadm binary to smithi196 2024-09-16T06:15:14.671 INFO:teuthology.orchestra.run.smithi190.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-16T06:15:15.270 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T06:15:15.458 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:15.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:15 smithi190 ceph-mon[25888]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:15.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:15.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:15.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:15.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:16.556 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:16 smithi190 ceph-mon[25888]: Added host smithi196 2024-09-16T06:15:16.556 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:17.231 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:15:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:[{"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-16T06:15:17.552 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:17 smithi190 ceph-mon[25888]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:17.841 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T06:15:17.842 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T06:15:18.029 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:18.431 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:18 smithi190 ceph-mon[25888]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:19.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:20.281 INFO:teuthology.orchestra.run.smithi190.stderr:adjusted tunables profile to default 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3100341215' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:20.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:20.879 INFO:tasks.cephadm:Adding mon.a on smithi190 2024-09-16T06:15:20.879 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-09-16T06:15:20.879 INFO:tasks.cephadm:Adding mon.b on smithi196 2024-09-16T06:15:20.879 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b' 2024-09-16T06:15:21.075 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:15:21.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:21 smithi190 ceph-mon[25888]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:15:21.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:21 smithi190 ceph-mon[25888]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:21.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:21 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3100341215' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T06:15:21.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:21 smithi190 ceph-mon[25888]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:22.704 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-16T06:15:23.305 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T06:15:23.308 DEBUG:teuthology.orchestra.run.smithi196:mon.b> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T06:15:23.310 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T06:15:23.310 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T06:15:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:23.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:23.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:23.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:23.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:23.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:24.041 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:15:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:24 smithi190 ceph-mon[25888]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:24 smithi190 ceph-mon[25888]: Saving service mon spec with placement smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-16T06:15:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:24 smithi190 ceph-mon[25888]: Deploying daemon mon.b on smithi196 2024-09-16T06:15:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:25 smithi190 ceph-mon[25888]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:27.275 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:15:27.275 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"bd41205c-73f2-11ef-bceb-c7b262605968","modified":"2024-09-16T06:14:04.955489Z","created":"2024-09-16T06:14:04.955489Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T06:15:27.275 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-16T06:15:27.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:27 smithi190 ceph-mon[25888]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:27.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:27 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/2765346327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T06:15:28.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 podman[30540]: 2024-09-16 06:15:28.043729598 +0000 UTC m=+0.114333808 container create 7dcb6335223adb6ece404b21bdbce2ed613bef1f933455f5b9443f3e02ab9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-16T06:15:28.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 podman[30540]: 2024-09-16 06:15:28.246015652 +0000 UTC m=+0.316619863 container init 7dcb6335223adb6ece404b21bdbce2ed613bef1f933455f5b9443f3e02ab9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T06:15:28.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 podman[30540]: 2024-09-16 06:15:28.250966393 +0000 UTC m=+0.321570606 container start 7dcb6335223adb6ece404b21bdbce2ed613bef1f933455f5b9443f3e02ab9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T06:15:28.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 7 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: pidfile_write: ignore empty --pid-file 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: load: jerasure load: lrc 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: RocksDB version: 7.9.2 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Git sha 0 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: DB SUMMARY 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: DB Session ID: UYRSCQCX7OCAKLX88P67 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: CURRENT file: CURRENT 2024-09-16T06:15:28.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.error_if_exists: 0 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.create_if_missing: 0 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.paranoid_checks: 1 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.env: 0x556832482bc0 2024-09-16T06:15:28.518 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T06:15:28.519 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.info_log: 0x5568335eebe0 2024-09-16T06:15:28.519 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.statistics: (nil) 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.use_fsync: 0 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_log_file_size: 0 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T06:15:28.521 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.allow_fallocate: 1 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.use_direct_reads: 0 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.db_log_dir: 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.wal_dir: 2024-09-16T06:15:28.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.write_buffer_manager: 0x5568335fc320 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T06:15:28.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.unordered_write: 0 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T06:15:28.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.row_cache: None 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.wal_filter: None 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.two_write_queues: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.wal_compression: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.atomic_flush: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.log_readahead_size: 0 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T06:15:28.525 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_background_jobs: 2 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_background_compactions: -1 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_subcompactions: 1 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T06:15:28.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_open_files: -1 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_background_flushes: -1 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Compression algorithms supported: 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kZSTD supported: 0 2024-09-16T06:15:28.527 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kXpressCompression supported: 0 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kBZip2Compression supported: 0 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kLZ4Compression supported: 1 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kZlibCompression supported: 1 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: kSnappyCompression supported: 1 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.merge_operator: 2024-09-16T06:15:28.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_filter: None 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5568335ee7e0) 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_type: 0 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_block_index_type: 0 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_shortening: 1 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T06:15:28.529 INFO:journalctl@ceph.mon.b.smithi196.stdout: checksum: 4 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: no_block_cache: 0 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache: 0x5568335e3350 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_options: 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: capacity : 536870912 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: num_shard_bits : 4 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: strict_capacity_limit : 0 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_compressed: (nil) 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: persistent_cache: (nil) 2024-09-16T06:15:28.530 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_size: 4096 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_size_deviation: 10 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_restart_interval: 16 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_block_restart_interval: 1 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: metadata_block_size: 4096 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: partition_filters: 0 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: use_delta_encoding: 1 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: filter_policy: bloomfilter 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: whole_key_filtering: 1 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: verify_compression: 0 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-09-16T06:15:28.531 INFO:journalctl@ceph.mon.b.smithi196.stdout: format_version: 5 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_index_compression: 1 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_align: 0 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout: max_auto_readahead_size: 262144 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout: prepopulate_block_cache: 0 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout: initial_auto_readahead_size: 8192 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T06:15:28.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression: NoCompression 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.num_levels: 7 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T06:15:28.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T06:15:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T06:15:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T06:15:28.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.inplace_update_support: 0 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T06:15:28.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.bloom_locality: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.max_successive_merges: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.ttl: 2592000 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.enable_blob_files: false 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.min_blob_size: 0 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T06:15:28.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ed370bf1-f060-41c4-97c9-c474f01329cf 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467328277561, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T06:15:28.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467328277966, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726467328, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ed370bf1-f060-41c4-97c9-c474f01329cf", "db_session_id": "UYRSCQCX7OCAKLX88P67", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467328278060, "job": 1, "event": "recovery_finished"} 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55683360ee00 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: DB pointer 0x556833702000 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** DB Stats ** 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T06:15:28.540 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:28.541 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T06:15:28.542 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: Block cache BinnedLRUCache@0x5568335e3350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-16T06:15:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: starting mon.b rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(???) e0 preinit fsid bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout: e1 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T06:15:28.544 INFO:journalctl@ceph.mon.b.smithi196.stdout: legacy client fscid: -1 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout: No filesystems configured 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:15:28.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/3100341215' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T06:15:28.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/3100341215' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:28.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: Saving service mon spec with placement smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: Deploying daemon mon.b on smithi196 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/2765346327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 ceph-mon[30578]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 bash[30540]: 7dcb6335223adb6ece404b21bdbce2ed613bef1f933455f5b9443f3e02ab9203 2024-09-16T06:15:28.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:28 smithi196 systemd[1]: Started Ceph mon.b for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:15:28.798 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T06:15:28.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T06:15:29.012 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:15:33.697 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: mon.a calling monitor election 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: mon.b calling monitor election 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: monmap e2: 2 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: fsmap 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: mgrmap e13: y(active, since 39s) 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: overall HEALTH_OK 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:33.699 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:33.700 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:33.757 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:15:33.758 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"bd41205c-73f2-11ef-bceb-c7b262605968","modified":"2024-09-16T06:15:28.307077Z","created":"2024-09-16T06:14:04.955489Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T06:15:33.758 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-16T06:15:33.758 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:33.758 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.758 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: mon.a calling monitor election 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: mon.b calling monitor election 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T06:15:33.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: monmap e2: 2 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: fsmap 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: mgrmap e13: y(active, since 39s) 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: overall HEALTH_OK 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:33.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:34.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:34 smithi196 ceph-mon[30578]: Deploying daemon mon.c on smithi190 2024-09-16T06:15:34.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:34 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/1823610859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T06:15:34.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:34.713 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:34 smithi190 ceph-mon[25888]: Deploying daemon mon.c on smithi190 2024-09-16T06:15:34.713 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:34 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/1823610859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T06:15:34.713 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:35.237 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T06:15:35.237 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T06:15:35.439 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:15:35.558 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:35 smithi196 ceph-mon[30578]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:35.811 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:35 smithi190 ceph-mon[25888]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:36.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 systemd[1]: Starting Ceph mon.c for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:15:36.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 podman[39693]: 2024-09-16 06:15:36.374897861 +0000 UTC m=+0.015339985 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:15:36.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 podman[39693]: 2024-09-16 06:15:36.487566755 +0000 UTC m=+0.128008866 container create 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T06:15:36.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 podman[39693]: 2024-09-16 06:15:36.689991531 +0000 UTC m=+0.330433648 container init 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T06:15:36.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 podman[39693]: 2024-09-16 06:15:36.694449621 +0000 UTC m=+0.334891743 container start 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=reef-e4e227b) 2024-09-16T06:15:36.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 7 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pidfile_write: ignore empty --pid-file 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: load: jerasure load: lrc 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: RocksDB version: 7.9.2 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Git sha 0 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: DB SUMMARY 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: DB Session ID: YDF5HMIHFWSKS26MG7JR 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: CURRENT file: CURRENT 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T06:15:36.724 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.error_if_exists: 0 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.create_if_missing: 0 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.paranoid_checks: 1 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.env: 0x559fe091abc0 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.info_log: 0x559fe201abe0 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T06:15:36.725 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.statistics: (nil) 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.use_fsync: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_log_file_size: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.allow_fallocate: 1 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.use_direct_reads: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.db_log_dir: 2024-09-16T06:15:36.726 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.wal_dir: 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.write_buffer_manager: 0x559fe2028320 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T06:15:36.727 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.unordered_write: 0 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.row_cache: None 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.wal_filter: None 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T06:15:36.729 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.two_write_queues: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.wal_compression: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.atomic_flush: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T06:15:36.730 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.log_readahead_size: 0 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_background_jobs: 2 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_background_compactions: -1 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_subcompactions: 1 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T06:15:36.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_open_files: -1 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_background_flushes: -1 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Compression algorithms supported: 2024-09-16T06:15:36.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kZSTD supported: 0 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kXpressCompression supported: 0 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kBZip2Compression supported: 0 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kLZ4Compression supported: 1 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kZlibCompression supported: 1 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: kSnappyCompression supported: 1 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.merge_operator: 2024-09-16T06:15:36.733 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_filter: None 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559fe201a7e0) 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks: 1 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T06:15:36.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_top_level_index_and_filter: 1 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_type: 0 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_index_type: 0 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_shortening: 1 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: checksum: 4 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: no_block_cache: 0 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache: 0x559fe200f1f0 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_name: BinnedLRUCache 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_options: 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: capacity : 536870912 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_shard_bits : 4 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: strict_capacity_limit : 0 2024-09-16T06:15:36.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: high_pri_pool_ratio: 0.000 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_compressed: (nil) 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: persistent_cache: (nil) 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size: 4096 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size_deviation: 10 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_restart_interval: 16 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_block_restart_interval: 1 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: metadata_block_size: 4096 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: partition_filters: 0 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: use_delta_encoding: 1 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: filter_policy: bloomfilter 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: whole_key_filtering: 1 2024-09-16T06:15:36.736 INFO:journalctl@ceph.mon.c.smithi190.stdout: verify_compression: 0 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: read_amp_bytes_per_bit: 0 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: format_version: 5 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_index_compression: 1 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_align: 0 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: max_auto_readahead_size: 262144 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: prepopulate_block_cache: 0 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: initial_auto_readahead_size: 8192 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression: NoCompression 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T06:15:36.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.num_levels: 7 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T06:15:36.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T06:15:36.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T06:15:36.741 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T06:15:36.741 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T06:15:36.741 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T06:15:36.741 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T06:15:36.741 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T06:15:36.742 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.inplace_update_support: 0 2024-09-16T06:15:36.743 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.bloom_locality: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.max_successive_merges: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.ttl: 2592000 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T06:15:36.744 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.enable_blob_files: false 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.min_blob_size: 0 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T06:15:36.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ba68c5aa-930b-4356-a53f-350f6a973047 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467336721172, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467336721542, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726467336, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ba68c5aa-930b-4356-a53f-350f6a973047", "db_session_id": "YDF5HMIHFWSKS26MG7JR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726467336721633, "job": 1, "event": "recovery_finished"} 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559fe203ae00 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: DB pointer 0x559fe212e000 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T06:15:36.746 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** DB Stats ** 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:36.747 INFO:journalctl@ceph.mon.c.smithi190.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T06:15:36.748 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache BinnedLRUCache@0x559fe200f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: using public_addrv [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-09-16T06:15:36.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: starting mon.c rank -1 at public addrs [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] at bind addrs [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:36.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(???) e0 preinit fsid bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:37.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T06:15:37.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T06:15:37.099 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T06:15:37.100 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.101 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/3100341215' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:15:37.102 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/3100341215' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:37.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Saving service mon spec with placement smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Deploying daemon mon.b on smithi196 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/2765346327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.a calling monitor election 2024-09-16T06:15:37.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.b calling monitor election 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: monmap e2: 2 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: fsmap 2024-09-16T06:15:37.105 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mgrmap e13: y(active, since 39s) 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: overall HEALTH_OK 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: Deploying daemon mon.c on smithi190 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/1823610859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:37.106 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 ceph-mon[39731]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T06:15:37.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 bash[39693]: 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 2024-09-16T06:15:37.107 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:36 smithi190 systemd[1]: Started Ceph mon.c for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:15:42.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: mon.a calling monitor election 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: mon.b calling monitor election 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: monmap e3: 3 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: fsmap 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: mgrmap e13: y(active, since 47s) 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: mon.c (rank 2) addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] is down (out of quorum) 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.099 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: mon.a calling monitor election 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: mon.b calling monitor election 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: monmap e3: 3 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: fsmap 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: mgrmap e13: y(active, since 47s) 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: mon.c (rank 2) addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] is down (out of quorum) 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:42.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:43.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:43.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:42 smithi190 ceph-mon[25888]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:43.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:43.242 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-16T06:15:43.242 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"bd41205c-73f2-11ef-bceb-c7b262605968","modified":"2024-09-16T06:15:36.747800Z","created":"2024-09-16T06:14:04.955489Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T06:15:43.243 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-16T06:15:43.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:43.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:42 smithi196 ceph-mon[30578]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:43.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:43.868 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T06:15:43.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: mon.a calling monitor election 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: mon.b calling monitor election 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.003 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: monmap e3: 3 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: fsmap 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: mgrmap e13: y(active, since 47s) 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-16T06:15:44.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: mon.c (rank 2) addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] is down (out of quorum) 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:44.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:44.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: mon.c calling monitor election 2024-09-16T06:15:44.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: mon.c calling monitor election 2024-09-16T06:15:44.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: mon.b calling monitor election 2024-09-16T06:15:44.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: mon.a calling monitor election 2024-09-16T06:15:44.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T06:15:44.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: monmap e3: 3 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: fsmap 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: mgrmap e13: y(active, since 49s) 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: Cluster is now healthy 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: overall HEALTH_OK 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: mon.c calling monitor election 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: mon.c calling monitor election 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: mon.b calling monitor election 2024-09-16T06:15:44.709 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: mon.a calling monitor election 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: monmap e3: 3 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: fsmap 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: mgrmap e13: y(active, since 49s) 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: Cluster is now healthy 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: overall HEALTH_OK 2024-09-16T06:15:44.710 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:44 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:44.741 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: mon.c calling monitor election 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: mon.c calling monitor election 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: mon.b calling monitor election 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: mon.a calling monitor election 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: monmap e3: 3 mons at {a=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: fsmap 2024-09-16T06:15:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T06:15:44.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: mgrmap e13: y(active, since 49s) 2024-09-16T06:15:44.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-16T06:15:44.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: Cluster is now healthy 2024-09-16T06:15:44.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: overall HEALTH_OK 2024-09-16T06:15:44.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:44 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T06:15:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T06:15:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T06:15:45.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:46.691 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:46 smithi190 ceph-mon[25888]: Reconfiguring mon.c (monmap changed)... 2024-09-16T06:15:46.691 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:46 smithi190 ceph-mon[25888]: Reconfiguring daemon mon.c on smithi190 2024-09-16T06:15:46.691 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:46 smithi190 ceph-mon[39731]: Reconfiguring mon.c (monmap changed)... 2024-09-16T06:15:46.691 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:46 smithi190 ceph-mon[39731]: Reconfiguring daemon mon.c on smithi190 2024-09-16T06:15:46.691 INFO:teuthology.orchestra.run.smithi190.stdout:# minimal ceph.conf for bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:46.691 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-09-16T06:15:46.692 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:15:46.692 INFO:teuthology.orchestra.run.smithi190.stdout: mon_host = [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-09-16T06:15:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:46 smithi196 ceph-mon[30578]: Reconfiguring mon.c (monmap changed)... 2024-09-16T06:15:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:46 smithi196 ceph-mon[30578]: Reconfiguring daemon mon.c on smithi190 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/6568158' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:47.659 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:47.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/6568158' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:47.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:47.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:47.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:47.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T06:15:47.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/6568158' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T06:15:47.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:47.938 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T06:15:47.938 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:47.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T06:15:47.965 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:47.965 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:48.030 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:15:48.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T06:15:48.058 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:15:48.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:15:48.124 INFO:tasks.cephadm:Adding mgr.y on smithi190 2024-09-16T06:15:48.124 INFO:tasks.cephadm:Adding mgr.x on smithi196 2024-09-16T06:15:48.124 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi190=y;smithi196=x' 2024-09-16T06:15:48.757 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:15:48.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:48 smithi196 ceph-mon[30578]: Reconfiguring mon.b (monmap changed)... 2024-09-16T06:15:48.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:48 smithi196 ceph-mon[30578]: Reconfiguring daemon mon.b on smithi196 2024-09-16T06:15:48.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:48 smithi190 ceph-mon[25888]: Reconfiguring mon.b (monmap changed)... 2024-09-16T06:15:48.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:48 smithi190 ceph-mon[25888]: Reconfiguring daemon mon.b on smithi196 2024-09-16T06:15:48.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:48 smithi190 ceph-mon[39731]: Reconfiguring mon.b (monmap changed)... 2024-09-16T06:15:48.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:48 smithi190 ceph-mon[39731]: Reconfiguring daemon mon.b on smithi196 2024-09-16T06:15:49.748 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.749 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:49.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:49.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:49.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:49.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:49.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:49.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.481 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-16T06:15:50.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T06:15:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T06:15:50.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T06:15:50.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T06:15:50.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:51.045 DEBUG:teuthology.orchestra.run.smithi196:mgr.x> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T06:15:51.048 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T06:15:51.048 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:15:51.048 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:15:51.064 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T06:15:51.064 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme4n1 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme4n1 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:15:07.103074602 +0000 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:04.715488725 +0000 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:04.715488725 +0000 2024-09-16T06:15:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:04.711488836 +0000 2024-09-16T06:15:51.121 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-16T06:15:51.184 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:15:51.184 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:15:51.184 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000219864 s, 2.3 MB/s 2024-09-16T06:15:51.186 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-16T06:15:51.242 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme3n1 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme3n1 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:15:07.092074909 +0000 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:04.580492499 +0000 2024-09-16T06:15:51.299 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:04.580492499 +0000 2024-09-16T06:15:51.300 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:04.574492667 +0000 2024-09-16T06:15:51.300 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-16T06:15:51.363 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:15:51.363 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:15:51.363 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000256153 s, 2.0 MB/s 2024-09-16T06:15:51.364 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-16T06:15:51.420 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme2n1 2024-09-16T06:15:51.475 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme2n1 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:15:07.080075243 +0000 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:04.435496553 +0000 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:04.435496553 +0000 2024-09-16T06:15:51.476 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:04.430496693 +0000 2024-09-16T06:15:51.476 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-16T06:15:51.539 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:15:51.539 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:15:51.539 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000262097 s, 2.0 MB/s 2024-09-16T06:15:51.540 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-16T06:15:51.596 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/nvme1n1 2024-09-16T06:15:51.651 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/nvme1n1 2024-09-16T06:15:51.651 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:51.651 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-16T06:15:51.651 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:51.651 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:51.652 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-16 06:15:07.069075549 +0000 2024-09-16T06:15:51.652 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-16 06:13:04.291500580 +0000 2024-09-16T06:15:51.652 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-16 06:13:04.291500580 +0000 2024-09-16T06:15:51.652 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-16 06:13:04.286500720 +0000 2024-09-16T06:15:51.652 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-16T06:15:51.714 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-16T06:15:51.714 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-16T06:15:51.714 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000238008 s, 2.2 MB/s 2024-09-16T06:15:51.716 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-16T06:15:51.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:51 smithi196 ceph-mon[30578]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi190=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:51.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:51 smithi196 ceph-mon[30578]: Saving service mgr spec with placement smithi190=y;smithi196=x;count:2 2024-09-16T06:15:51.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:51 smithi196 ceph-mon[30578]: Deploying daemon mgr.x on smithi196 2024-09-16T06:15:51.772 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:15:51.772 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:15:51.789 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T06:15:51.789 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme4n1 2024-09-16T06:15:51.844 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme4n1 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:15:18.008359628 +0000 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:06.330068179 +0000 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:06.330068179 +0000 2024-09-16T06:15:51.845 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:06.325068206 +0000 2024-09-16T06:15:51.845 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-16T06:15:51.846 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:51 smithi190 ceph-mon[25888]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi190=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:51.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:51 smithi190 ceph-mon[25888]: Saving service mgr spec with placement smithi190=y;smithi196=x;count:2 2024-09-16T06:15:51.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:51 smithi190 ceph-mon[25888]: Deploying daemon mgr.x on smithi196 2024-09-16T06:15:51.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:51 smithi190 ceph-mon[39731]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi190=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:51.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:51 smithi190 ceph-mon[39731]: Saving service mgr spec with placement smithi190=y;smithi196=x;count:2 2024-09-16T06:15:51.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:51 smithi190 ceph-mon[39731]: Deploying daemon mgr.x on smithi196 2024-09-16T06:15:51.908 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:15:51.908 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:15:51.908 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000208647 s, 2.5 MB/s 2024-09-16T06:15:51.909 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-16T06:15:51.965 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme3n1 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme3n1 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:15:17.998359679 +0000 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:06.187068956 +0000 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:06.187068956 +0000 2024-09-16T06:15:52.021 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:06.182068983 +0000 2024-09-16T06:15:52.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-16T06:15:52.087 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:15:52.088 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:15:52.088 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000214142 s, 2.4 MB/s 2024-09-16T06:15:52.089 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-16T06:15:52.146 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme2n1 2024-09-16T06:15:52.209 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme2n1 2024-09-16T06:15:52.209 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:52.209 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-16T06:15:52.209 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:52.209 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:52.209 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:15:17.988359729 +0000 2024-09-16T06:15:52.210 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:06.043069738 +0000 2024-09-16T06:15:52.210 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:06.043069738 +0000 2024-09-16T06:15:52.210 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:06.038069765 +0000 2024-09-16T06:15:52.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-16T06:15:52.322 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:15:52.323 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:15:52.323 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000219128 s, 2.3 MB/s 2024-09-16T06:15:52.324 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-16T06:15:52.339 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme1n1 2024-09-16T06:15:52.401 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme1n1 2024-09-16T06:15:52.401 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T06:15:52.401 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-16T06:15:52.401 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T06:15:52.401 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T06:15:52.402 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-16 06:15:17.978359780 +0000 2024-09-16T06:15:52.402 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-16 06:13:05.904070494 +0000 2024-09-16T06:15:52.402 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-16 06:13:05.904070494 +0000 2024-09-16T06:15:52.402 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-16 06:13:05.899070521 +0000 2024-09-16T06:15:52.402 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-16T06:15:52.513 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-16T06:15:52.514 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-16T06:15:52.514 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000288564 s, 1.8 MB/s 2024-09-16T06:15:52.515 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-16T06:15:52.531 INFO:tasks.cephadm:Deploying osd.0 on smithi190 with /dev/nvme1n1... 2024-09-16T06:15:52.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-16T06:15:52.721 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:52.745 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:52 smithi196 ceph-mon[30578]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:52.745 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:52 smithi196 systemd[1]: Starting Ceph mgr.x for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:15:52.842 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:52 smithi190 ceph-mon[25888]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:52.842 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:52 smithi190 ceph-mon[39731]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:53.016 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:52 smithi196 podman[32642]: 2024-09-16 06:15:52.647785637 +0000 UTC m=+0.015477520 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:15:53.016 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:52 smithi196 podman[32642]: 2024-09-16 06:15:52.758515928 +0000 UTC m=+0.126207812 container create 33449789fc6c3fd25bb46a33cb024ea2f1800659b4b4733cb9bf444f84716b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T06:15:53.016 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:52 smithi196 podman[32642]: 2024-09-16 06:15:52.952883878 +0000 UTC m=+0.320575767 container init 33449789fc6c3fd25bb46a33cb024ea2f1800659b4b4733cb9bf444f84716b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T06:15:53.016 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:52 smithi196 podman[32642]: 2024-09-16 06:15:52.959342059 +0000 UTC m=+0.327033944 container start 33449789fc6c3fd25bb46a33cb024ea2f1800659b4b4733cb9bf444f84716b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:15:53.016 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:52 smithi196 ceph-mgr[32680]: -- 172.21.15.196:0/3941374401 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ecf786d00 con 0x558ecf75b000 2024-09-16T06:15:53.363 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 bash[32642]: 33449789fc6c3fd25bb46a33cb024ea2f1800659b4b4733cb9bf444f84716b1b 2024-09-16T06:15:53.363 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 systemd[1]: Started Ceph mgr.x for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:15:53.363 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:53.077+0000 7fc455991040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:15:53.363 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:53.138+0000 7fc455991040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:15:53.363 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:53.294+0000 7fc455991040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:15:53.363 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:53.362+0000 7fc455991040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:15:53.677 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:53.531+0000 7fc455991040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:15:53.677 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:53.676+0000 7fc455991040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:15:54.250 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.250 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.250 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.250 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.250 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:54.251 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:54.001+0000 7fc455991040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:15:54.251 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:54.063+0000 7fc455991040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:15:54.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.348 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:54.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:54.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:54.515 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:54.249+0000 7fc455991040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:15:54.576 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/nvme1n1 2024-09-16T06:15:54.577 INFO:teuthology.orchestra.run.smithi190.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:15:54.581 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-16T06:15:54.598 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-09-16T06:15:54.598 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-09-16T06:15:54.598 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152244 s, 689 MB/s 2024-09-16T06:15:54.598 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 2024-09-16T06:15:54.598 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-09-16T06:15:55.515 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:55.133+0000 7fc455991040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:15:55.515 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:55.198+0000 7fc455991040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T06:15:55.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T06:15:55.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:55.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-mon[30578]: Reconfiguring daemon mgr.y on smithi190 2024-09-16T06:15:55.532 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:55.532 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.532 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[25888]: Reconfiguring daemon mgr.y on smithi190 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:55.533 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:55.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:55 smithi190 ceph-mon[39731]: Reconfiguring daemon mgr.y on smithi190 2024-09-16T06:15:55.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:/dev/nvme1n1 2024-09-16T06:15:56.162 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:55 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:55.909+0000 7fc455991040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T06:15:56.163 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.022+0000 7fc455991040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:15:56.163 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.162+0000 7fc455991040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:15:56.306 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:15:56.443 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.222+0000 7fc455991040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:15:56.443 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.371+0000 7fc455991040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T06:15:56.443 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.442+0000 7fc455991040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:15:56.722 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.502+0000 7fc455991040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:15:56.722 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.658+0000 7fc455991040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:15:56.722 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.721+0000 7fc455991040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:15:56.983 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.786+0000 7fc455991040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:15:56.983 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.867+0000 7fc455991040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:15:56.984 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.925+0000 7fc455991040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:15:56.984 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:56.982+0000 7fc455991040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:57.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:57.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:57.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:57.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:57.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:57.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:15:57.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:57.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:15:57.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:15:58.015 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:15:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:15:57.758+0000 7fc455991040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: Standby manager daemon x started 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:15:58.647 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: Standby manager daemon x started 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:15:58.648 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:58 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: Standby manager daemon x started 2024-09-16T06:15:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T06:15:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T06:15:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T06:15:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/1915652331' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T06:15:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:15:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:15:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:58 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:15:59.631 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[25888]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[25888]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[25888]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[39731]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[39731]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[39731]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-16T06:15:59.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:15:59 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T06:15:59.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:59 smithi196 ceph-mon[30578]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:15:59.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:59 smithi196 ceph-mon[30578]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:15:59.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:59 smithi196 ceph-mon[30578]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-16T06:15:59.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:15:59 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T06:16:00.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:00 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/950167920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]: dispatch 2024-09-16T06:16:00.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:00 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]: dispatch 2024-09-16T06:16:00.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:00 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]': finished 2024-09-16T06:16:00.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:00 smithi196 ceph-mon[30578]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T06:16:00.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:00 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/950167920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]: dispatch 2024-09-16T06:16:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]: dispatch 2024-09-16T06:16:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]': finished 2024-09-16T06:16:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[25888]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T06:16:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:00.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/950167920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]: dispatch 2024-09-16T06:16:00.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]: dispatch 2024-09-16T06:16:00.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0a4292e-21dc-43b1-9cfa-0d5de57a2484"}]': finished 2024-09-16T06:16:00.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[39731]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T06:16:00.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:00 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:01.590 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:01 smithi190 ceph-mon[25888]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:01.590 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:01 smithi190 ceph-mon[39731]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:01.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:01 smithi196 ceph-mon[30578]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:02.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:02 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/918216728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:02.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:02 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/918216728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:02.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:02 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/918216728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:03.734 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:03 smithi190 ceph-mon[25888]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:03.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:03 smithi190 ceph-mon[39731]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:03 smithi196 ceph-mon[30578]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:05.736 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:05 smithi190 ceph-mon[25888]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:05.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:05 smithi190 ceph-mon[39731]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:05.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:05 smithi196 ceph-mon[30578]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:07.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:07 smithi190 ceph-mon[39731]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:07.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:07 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T06:16:07.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:07 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:07.495 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:07 smithi190 ceph-mon[25888]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:07.496 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:07 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T06:16:07.496 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:07 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:07.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:07 smithi196 ceph-mon[30578]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:07.826 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:07 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T06:16:07.826 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:07 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:08.539 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:08 smithi190 ceph-mon[25888]: Deploying daemon osd.0 on smithi190 2024-09-16T06:16:08.540 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:08 smithi190 ceph-mon[39731]: Deploying daemon osd.0 on smithi190 2024-09-16T06:16:08.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:08 smithi196 ceph-mon[30578]: Deploying daemon osd.0 on smithi190 2024-09-16T06:16:09.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:09 smithi196 ceph-mon[30578]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:09.776 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:09 smithi190 ceph-mon[25888]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:09 smithi190 ceph-mon[39731]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:11.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:11 smithi190 ceph-mon[25888]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:11.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:11 smithi190 ceph-mon[39731]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:11.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:11 smithi196 ceph-mon[30578]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:12.661 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:12.661 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:12.661 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:12.661 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:12.662 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:12.662 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:12.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:12.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:12.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:13.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:13 smithi196 ceph-mon[30578]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:13.841 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:13 smithi190 ceph-mon[25888]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:13.841 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:13 smithi190 ceph-mon[39731]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[25888]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:15 smithi190 ceph-mon[39731]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T06:16:15.633 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 0 on host 'smithi190' 2024-09-16T06:16:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:15.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:15.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:15.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:15 smithi196 ceph-mon[30578]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T06:16:16.231 DEBUG:teuthology.orchestra.run.smithi190:osd.0> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T06:16:16.233 INFO:tasks.cephadm:Deploying osd.1 on smithi190 with /dev/nvme2n1... 2024-09-16T06:16:16.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:16.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[39731]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:16.527 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:16 smithi190 ceph-mon[25888]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:16 smithi196 ceph-mon[30578]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:16:16.970 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:16:17.347 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0[43985]: 2024-09-16T06:16:17.053+0000 7f87e54d5640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T06:16:17.347 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0[43985]: 2024-09-16T06:16:17.057+0000 7f87e12fe640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:16:17.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:17 smithi196 ceph-mon[30578]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:17.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:17 smithi196 ceph-mon[30578]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:16:17.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:17 smithi196 ceph-mon[30578]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T06:16:17.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:17.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[25888]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[25888]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[25888]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[39731]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:17.788 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[39731]: from='osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:16:17.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[39731]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T06:16:17.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:17.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[39731]: osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509] boot 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[39731]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:18.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:16:18.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:16:18.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:18.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[25888]: osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509] boot 2024-09-16T06:16:18.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[25888]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T06:16:18.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:18 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:18.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:18 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:16:18.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:18 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:16:18.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:18 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:18.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:18 smithi196 ceph-mon[30578]: osd.0 [v2:172.21.15.190:6802/62621509,v1:172.21.15.190:6803/62621509] boot 2024-09-16T06:16:18.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:18 smithi196 ceph-mon[30578]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T06:16:18.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:18 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:16:19.552 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:19 smithi190 ceph-mon[25888]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:19.553 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:19 smithi190 ceph-mon[25888]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T06:16:19.553 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:19 smithi190 ceph-mon[39731]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:19.553 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:19 smithi190 ceph-mon[39731]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T06:16:19.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:19 smithi196 ceph-mon[30578]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T06:16:19.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:19 smithi196 ceph-mon[30578]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T06:16:19.768 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/nvme2n1 2024-09-16T06:16:19.768 INFO:teuthology.orchestra.run.smithi190.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:16:19.773 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-16T06:16:19.790 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-09-16T06:16:19.790 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-09-16T06:16:19.790 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138869 s, 755 MB/s 2024-09-16T06:16:19.790 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-09-16T06:16:20.733 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:/dev/nvme2n1 2024-09-16T06:16:20.927 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:16:21.306 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi190 2024-09-16T06:16:21.306 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi190 to 16064M 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[25888]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi190 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.307 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi190 to 16064M 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.308 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:21 smithi190 ceph-mon[39731]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:21.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi190 2024-09-16T06:16:21.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi190 to 16064M 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:21 smithi196 ceph-mon[30578]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[25888]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:16:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:16:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:23.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[39731]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:23.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:16:23.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:16:23.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:23 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:23 smithi196 ceph-mon[30578]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:23 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:16:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:23 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:16:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:23 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:24 smithi190 ceph-mon[25888]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:16:24.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:24 smithi190 ceph-mon[39731]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:16:24.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:24 smithi196 ceph-mon[30578]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[25888]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2999595014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52400e70-bb15-46cf-9280-8b9c44a9c16b"}]: dispatch 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2999595014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52400e70-bb15-46cf-9280-8b9c44a9c16b"}]': finished 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[25888]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[39731]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2999595014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52400e70-bb15-46cf-9280-8b9c44a9c16b"}]: dispatch 2024-09-16T06:16:25.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2999595014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52400e70-bb15-46cf-9280-8b9c44a9c16b"}]': finished 2024-09-16T06:16:25.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[39731]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T06:16:25.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:25 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:25.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:25 smithi196 ceph-mon[30578]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:25.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:25 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2999595014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52400e70-bb15-46cf-9280-8b9c44a9c16b"}]: dispatch 2024-09-16T06:16:25.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:25 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2999595014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52400e70-bb15-46cf-9280-8b9c44a9c16b"}]': finished 2024-09-16T06:16:25.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:25 smithi196 ceph-mon[30578]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T06:16:25.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:25 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:26.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:26 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3838435296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:26.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:26 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/3838435296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:26.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:26 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/3838435296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:27.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:27 smithi196 ceph-mon[30578]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:27.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:27 smithi190 ceph-mon[25888]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:27.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:27 smithi190 ceph-mon[39731]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:29 smithi190 ceph-mon[25888]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:29.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:29 smithi190 ceph-mon[39731]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:29.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:29 smithi196 ceph-mon[30578]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:31.659 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:31 smithi190 ceph-mon[25888]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:31.659 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:31 smithi190 ceph-mon[39731]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:31.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:31 smithi196 ceph-mon[30578]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:32.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:32 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T06:16:32.698 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:32 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:32.699 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:32 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T06:16:32.699 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:32 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:32.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:32 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T06:16:32.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:32 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:33.541 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:33 smithi190 ceph-mon[25888]: Deploying daemon osd.1 on smithi190 2024-09-16T06:16:33.542 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:33 smithi190 ceph-mon[25888]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:33.542 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:33 smithi190 ceph-mon[39731]: Deploying daemon osd.1 on smithi190 2024-09-16T06:16:33.542 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:33 smithi190 ceph-mon[39731]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:33.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:33 smithi196 ceph-mon[30578]: Deploying daemon osd.1 on smithi190 2024-09-16T06:16:33.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:33 smithi196 ceph-mon[30578]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:35.508 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:35 smithi190 ceph-mon[25888]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:35.508 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:35 smithi190 ceph-mon[39731]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:35.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:35 smithi196 ceph-mon[30578]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:37.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:37 smithi196 ceph-mon[30578]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:37 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:37 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:37 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:37.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[25888]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:37.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:37.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:37.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:37.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[39731]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:37.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:37.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:37.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:37 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:39 smithi196 ceph-mon[30578]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:39.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:39.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:39.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.814 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[25888]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:39.814 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.814 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.814 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:39.814 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:39.814 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[39731]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:39.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:39.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:39.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:39.815 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:40.148 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 1 on host 'smithi190' 2024-09-16T06:16:40.718 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[25888]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T06:16:40.718 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:40.718 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:40.718 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:40.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[39731]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T06:16:40.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:40.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:40.719 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:40 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:40.739 DEBUG:teuthology.orchestra.run.smithi190:osd.1> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T06:16:40.741 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/nvme3n1... 2024-09-16T06:16:40.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-16T06:16:40.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:40 smithi196 ceph-mon[30578]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T06:16:40.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:40 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:16:40.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:40 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:40.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:40 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:41.496 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:16:41.640 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[25888]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:41.640 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[25888]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T06:16:41.640 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[25888]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T06:16:41.640 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:41.640 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[25888]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:16:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[39731]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[39731]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T06:16:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[39731]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T06:16:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-mon[39731]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:16:41.641 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1[52202]: 2024-09-16T06:16:41.378+0000 7f26c5fc9640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T06:16:41.641 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:16:41 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1[52202]: 2024-09-16T06:16:41.381+0000 7f26c1df2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:16:41.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:41 smithi196 ceph-mon[30578]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:41.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:41 smithi196 ceph-mon[30578]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T06:16:41.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:41 smithi196 ceph-mon[30578]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T06:16:41.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:41.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:41 smithi196 ceph-mon[30578]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:16:42.666 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:42 smithi196 ceph-mon[30578]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:16:42.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:42 smithi196 ceph-mon[30578]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T06:16:42.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:42.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:42.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[25888]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:16:42.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[25888]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T06:16:42.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:42.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:42.722 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[39731]: from='osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:16:42.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[39731]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T06:16:42.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:42.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:42 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:43.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:16:43.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:16:43.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[25888]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:43.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[25888]: osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266] boot 2024-09-16T06:16:43.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[25888]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T06:16:43.722 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:43.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:16:43.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:16:43.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[39731]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:43.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[39731]: osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266] boot 2024-09-16T06:16:43.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[39731]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T06:16:43.723 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:43.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:43 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:16:43.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:43 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:16:43.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:43 smithi196 ceph-mon[30578]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T06:16:43.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:43 smithi196 ceph-mon[30578]: osd.1 [v2:172.21.15.190:6810/525777266,v1:172.21.15.190:6811/525777266] boot 2024-09-16T06:16:43.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:43 smithi196 ceph-mon[30578]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T06:16:43.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:43 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:16:44.205 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/nvme3n1 2024-09-16T06:16:44.205 INFO:teuthology.orchestra.run.smithi190.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:16:44.205 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-16T06:16:44.211 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-09-16T06:16:44.211 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-09-16T06:16:44.211 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136954 s, 766 MB/s 2024-09-16T06:16:44.212 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-09-16T06:16:44.567 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:44 smithi190 ceph-mon[25888]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T06:16:44.567 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:44 smithi190 ceph-mon[39731]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T06:16:44.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:44 smithi196 ceph-mon[30578]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T06:16:45.245 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:/dev/nvme3n1 2024-09-16T06:16:45.444 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.565 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:45.566 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:45 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:16:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:16:45.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:45 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:16:46.576 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:46 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi190 2024-09-16T06:16:46.576 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:46 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi190 to 8032M 2024-09-16T06:16:46.576 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:46 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi190 2024-09-16T06:16:46.576 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:46 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi190 to 8032M 2024-09-16T06:16:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:46 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi190 2024-09-16T06:16:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:46 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi190 to 8032M 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[25888]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[39731]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:16:47.720 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:16:47.721 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:47.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:47 smithi196 ceph-mon[30578]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:47.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:16:47.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:16:47.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:48.579 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:48 smithi190 ceph-mon[25888]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:16:48.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:48 smithi190 ceph-mon[39731]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:16:48.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:48 smithi196 ceph-mon[30578]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:16:49.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:49 smithi190 ceph-mon[25888]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:49.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:49 smithi190 ceph-mon[39731]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:49 smithi196 ceph-mon[30578]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:50.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:50 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/645157673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]: dispatch 2024-09-16T06:16:50.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:50 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]: dispatch 2024-09-16T06:16:50.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:50 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]': finished 2024-09-16T06:16:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:50 smithi196 ceph-mon[30578]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T06:16:50.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:16:50.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/645157673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]: dispatch 2024-09-16T06:16:50.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]: dispatch 2024-09-16T06:16:50.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]': finished 2024-09-16T06:16:50.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[39731]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T06:16:50.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:16:50.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/645157673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]: dispatch 2024-09-16T06:16:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]: dispatch 2024-09-16T06:16:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46b8a2a4-b143-4784-8780-0ed28f8e0276"}]': finished 2024-09-16T06:16:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[25888]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T06:16:50.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:16:51.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:51 smithi196 ceph-mon[30578]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:51.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:51 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/1126310590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:51.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:51 smithi190 ceph-mon[39731]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:51.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:51 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/1126310590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:51.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:51 smithi190 ceph-mon[25888]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:51.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:51 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1126310590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:16:53.756 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:53 smithi190 ceph-mon[25888]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:53.756 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:53 smithi190 ceph-mon[39731]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:53.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:53 smithi196 ceph-mon[30578]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:55.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:55 smithi190 ceph-mon[25888]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:55.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:55 smithi190 ceph-mon[39731]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:55.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:55 smithi196 ceph-mon[30578]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:57.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:57 smithi190 ceph-mon[25888]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:57.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T06:16:57.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:57 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:57.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:57 smithi190 ceph-mon[39731]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:57.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T06:16:57.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:57 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:57.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:57 smithi196 ceph-mon[30578]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:57.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T06:16:57.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:57 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:16:58.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:58 smithi190 ceph-mon[39731]: Deploying daemon osd.2 on smithi190 2024-09-16T06:16:58.740 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:58 smithi190 ceph-mon[25888]: Deploying daemon osd.2 on smithi190 2024-09-16T06:16:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:58 smithi196 ceph-mon[30578]: Deploying daemon osd.2 on smithi190 2024-09-16T06:16:59.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:16:59 smithi196 ceph-mon[30578]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:59.779 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:16:59 smithi190 ceph-mon[25888]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:16:59.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:16:59 smithi190 ceph-mon[39731]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:01.534 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:01 smithi190 ceph-mon[25888]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:01.534 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:01 smithi190 ceph-mon[39731]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:01.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:01 smithi196 ceph-mon[30578]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:02.580 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:02 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:02.580 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:02 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:02.580 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:02 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:02.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:02 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:02.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:02 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:02.581 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:02 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:02.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:02 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:02.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:02 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:02.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:02 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:03.513 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:03 smithi190 ceph-mon[25888]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:03.513 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:03 smithi190 ceph-mon[39731]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:03 smithi196 ceph-mon[30578]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:04.943 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-09-16T06:17:05.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:05 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.522 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.523 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:05 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:05.544 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T06:17:05.546 INFO:tasks.cephadm:Deploying osd.3 on smithi190 with /dev/nvme4n1... 2024-09-16T06:17:05.547 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-16T06:17:06.217 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:17:06.480 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[25888]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T06:17:06.480 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[25888]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T06:17:06.480 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:06.480 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[25888]: from='osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:06.480 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[25888]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:06.481 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[39731]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T06:17:06.481 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[39731]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T06:17:06.481 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:06.481 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[39731]: from='osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:06.481 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-mon[39731]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:06.481 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2[60754]: 2024-09-16T06:17:06.233+0000 7f10f67ef640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T06:17:06.481 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:06 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2[60754]: 2024-09-16T06:17:06.237+0000 7f10f2618640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:17:06.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:06 smithi196 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T06:17:06.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:06 smithi196 ceph-mon[30578]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T06:17:06.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:06.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:06 smithi196 ceph-mon[30578]: from='osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:06.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:06 smithi196 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:07.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:07 smithi196 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:17:07.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:07 smithi196 ceph-mon[30578]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T06:17:07.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:07 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:07.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:07 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:07.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:07 smithi196 ceph-mon[30578]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[25888]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[25888]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[25888]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[39731]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[39731]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:07.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:07.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:07 smithi190 ceph-mon[39731]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[25888]: osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404] boot 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[25888]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:17:08.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:08.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[39731]: osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404] boot 2024-09-16T06:17:08.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[39731]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T06:17:08.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:08.381 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/nvme4n1 2024-09-16T06:17:08.381 INFO:teuthology.orchestra.run.smithi190.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:17:08.387 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-16T06:17:08.403 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-09-16T06:17:08.403 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-09-16T06:17:08.404 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134314 s, 781 MB/s 2024-09-16T06:17:08.404 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-09-16T06:17:08.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:08 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:17:08.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:08 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:17:08.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:08.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:08 smithi196 ceph-mon[30578]: osd.2 [v2:172.21.15.190:6818/3137551404,v1:172.21.15.190:6819/3137551404] boot 2024-09-16T06:17:08.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:08 smithi196 ceph-mon[30578]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T06:17:08.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:17:09.289 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi190:/dev/nvme4n1 2024-09-16T06:17:09.514 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:17:09.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:09 smithi196 ceph-mon[30578]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T06:17:09.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:09 smithi196 ceph-mon[30578]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:09.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:09 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T06:17:09.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:09 smithi190 ceph-mon[39731]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T06:17:09.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:09 smithi190 ceph-mon[39731]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:09.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:09 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T06:17:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:09 smithi190 ceph-mon[25888]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T06:17:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:09 smithi190 ceph-mon[25888]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:09.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:09 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T06:17:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T06:17:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:10 smithi190 ceph-mon[25888]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T06:17:10.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:10 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T06:17:10.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:10 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T06:17:10.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:10 smithi190 ceph-mon[39731]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T06:17:10.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:10 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T06:17:10.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:10 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T06:17:10.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:10 smithi196 ceph-mon[30578]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T06:17:10.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:10 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T06:17:11.057 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:10 smithi190 sudo[67039]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-16T06:17:11.057 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:10 smithi190 sudo[67039]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:11.057 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:10 smithi190 sudo[67039]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:11.057 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:10 smithi190 sudo[67039]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[25888]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[25888]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[39731]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[39731]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:11.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:11.348 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67062]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-16T06:17:11.348 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67062]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:11.348 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67062]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:11.348 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67062]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:11.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T06:17:11.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:11 smithi196 ceph-mon[30578]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T06:17:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:11 smithi196 ceph-mon[30578]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:11.719 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-16T06:17:11.720 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:11.720 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67065]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:11.720 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67065]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:12.005 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67126]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-16T06:17:12.005 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67126]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:12.005 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67126]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:12.005 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:17:11 smithi190 sudo[67126]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:12.259 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-16T06:17:12.259 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:12.259 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67196]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:12.260 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67196]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:12.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:12.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:12 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:12.542 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67254]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-16T06:17:12.542 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67254]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:12.542 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67254]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:12.542 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67254]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:12.543 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:12.544 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:12.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:12.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:12 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:12.801 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T06:17:12.801 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:12.801 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67276]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:12.801 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67276]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:13.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T06:17:13.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:13.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:13.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:12 smithi190 sudo[67297]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:13.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33330]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T06:17:13.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33330]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:13.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33330]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:13.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33330]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi190 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi190 to 5354M 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:13.577 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi190 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi190 to 5354M 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:13.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:13.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:13.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33333]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33333]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33333]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 sudo[33333]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi190 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi190 to 5354M 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:13.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:13.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:13.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:13.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:13.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67342]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T06:17:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67342]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67342]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67342]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:14.333 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67362]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T06:17:14.333 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67362]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T06:17:14.333 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67362]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T06:17:14.334 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:13 smithi190 sudo[67362]: pam_unix(sudo:session): session closed for user root 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:14.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2571499274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cbdf234-9282-4ab1-a636-03abcecece3c"}]: dispatch 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2571499274' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cbdf234-9282-4ab1-a636-03abcecece3c"}]': finished 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:14.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:14.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:14.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:14.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2571499274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cbdf234-9282-4ab1-a636-03abcecece3c"}]: dispatch 2024-09-16T06:17:14.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2571499274' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cbdf234-9282-4ab1-a636-03abcecece3c"}]': finished 2024-09-16T06:17:14.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T06:17:14.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:14 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:14.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:14.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:14.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2571499274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cbdf234-9282-4ab1-a636-03abcecece3c"}]: dispatch 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2571499274' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cbdf234-9282-4ab1-a636-03abcecece3c"}]': finished 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T06:17:14.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:14 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:15.497 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:15 smithi190 ceph-mon[25888]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:15.497 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:15 smithi190 ceph-mon[39731]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:15 smithi196 ceph-mon[30578]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:16.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:16 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/4194263551' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:17:16.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:16 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/4194263551' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:17:16.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:16 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/4194263551' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:17:17.641 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:17 smithi190 ceph-mon[25888]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:17 smithi190 ceph-mon[39731]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:17.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:17 smithi196 ceph-mon[30578]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:19.670 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:19 smithi190 ceph-mon[25888]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:19.670 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:19 smithi190 ceph-mon[39731]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:19.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:19 smithi196 ceph-mon[30578]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:21.689 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:21 smithi190 ceph-mon[25888]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T06:17:21.689 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T06:17:21.689 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:21.689 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:21 smithi190 ceph-mon[39731]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T06:17:21.689 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T06:17:21.690 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:21.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:21 smithi196 ceph-mon[30578]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T06:17:21.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T06:17:21.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:22.670 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:22 smithi190 ceph-mon[25888]: Deploying daemon osd.3 on smithi190 2024-09-16T06:17:22.670 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:22 smithi190 ceph-mon[39731]: Deploying daemon osd.3 on smithi190 2024-09-16T06:17:22.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:22 smithi196 ceph-mon[30578]: Deploying daemon osd.3 on smithi190 2024-09-16T06:17:23.737 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:23 smithi190 ceph-mon[25888]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T06:17:23.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:23 smithi190 ceph-mon[39731]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T06:17:23.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:23 smithi196 ceph-mon[30578]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T06:17:25.481 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:25 smithi190 ceph-mon[25888]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T06:17:25.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:25 smithi190 ceph-mon[39731]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T06:17:25.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:25 smithi196 ceph-mon[30578]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T06:17:26.630 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:26 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:26.630 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:26 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:26.630 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:26 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:26 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:26 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:26 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:26.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:26 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:26.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:26 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:26.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:26 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:27.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:27 smithi190 ceph-mon[25888]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 769 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-16T06:17:27.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:27 smithi190 ceph-mon[39731]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 769 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-16T06:17:27.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:27 smithi196 ceph-mon[30578]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 769 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-16T06:17:29.288 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 3 on host 'smithi190' 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[25888]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[25888]: from='osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[25888]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[39731]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[39731]: from='osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T06:17:29.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[39731]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T06:17:29.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:29.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:29.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:29 smithi196 ceph-mon[30578]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T06:17:29.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:29 smithi196 ceph-mon[30578]: from='osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T06:17:29.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:29 smithi196 ceph-mon[30578]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T06:17:29.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:29.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:29.885 DEBUG:teuthology.orchestra.run.smithi190:osd.3> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T06:17:29.888 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/nvme1n1... 2024-09-16T06:17:29.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-16T06:17:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: from='osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:30.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:30 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: from='osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T06:17:30.708 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T06:17:30.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:30.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: from='osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:30.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-09-16T06:17:30.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:30.709 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:30.984 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:17:30.852+0000 7f8a7c3da640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T06:17:30.984 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:17:30 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:17:30.856+0000 7f8a739e4640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:17:31.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[39731]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:31.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[39731]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:17:31.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[39731]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T06:17:31.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:31.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:31.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[39731]: from='osd.3 ' entity='osd.3' 2024-09-16T06:17:31.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[25888]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:31.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[25888]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:17:31.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[25888]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T06:17:31.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:31.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:31.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:31 smithi190 ceph-mon[25888]: from='osd.3 ' entity='osd.3' 2024-09-16T06:17:31.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:31 smithi196 ceph-mon[30578]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:31.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:31 smithi196 ceph-mon[30578]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-09-16T06:17:31.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:31 smithi196 ceph-mon[30578]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T06:17:31.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:31 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:31.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:31 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:31.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:31 smithi196 ceph-mon[30578]: from='osd.3 ' entity='osd.3' 2024-09-16T06:17:31.790 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme1n1 2024-09-16T06:17:31.791 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:17:31.795 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-16T06:17:31.810 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-16T06:17:31.811 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-16T06:17:31.811 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012918 s, 812 MB/s 2024-09-16T06:17:31.811 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-16T06:17:32.416 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme1n1 2024-09-16T06:17:32.609 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:17:32.728 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:32 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:17:32.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:32 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:17:32.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:32 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:32.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:32 smithi196 ceph-mon[30578]: osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168] boot 2024-09-16T06:17:32.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:32 smithi196 ceph-mon[30578]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T06:17:32.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:32 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[39731]: osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168] boot 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[39731]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:17:32.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:17:32.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:32.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[25888]: osd.3 [v2:172.21.15.190:6826/1507636168,v1:172.21.15.190:6827/1507636168] boot 2024-09-16T06:17:32.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[25888]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T06:17:32.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:32 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:17:33.587 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:33 smithi196 ceph-mon[30578]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:33.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:33 smithi196 ceph-mon[30578]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T06:17:33.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:33 smithi190 ceph-mon[25888]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:33.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:33 smithi190 ceph-mon[25888]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T06:17:33.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:33 smithi190 ceph-mon[39731]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-16T06:17:33.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:33 smithi190 ceph-mon[39731]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T06:17:35.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi190 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi190 to 4016M 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:34 smithi196 ceph-mon[30578]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:35.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi190 2024-09-16T06:17:35.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi190 to 4016M 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:35.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[39731]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi190 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.349 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi190 to 4016M 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:17:35.350 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:35.351 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:35.351 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:35.351 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:34 smithi190 ceph-mon[25888]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:37.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[25888]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:37.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/3683244918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]: dispatch 2024-09-16T06:17:37.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]: dispatch 2024-09-16T06:17:37.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]': finished 2024-09-16T06:17:37.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[25888]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[39731]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/3683244918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]: dispatch 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]: dispatch 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]': finished 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[39731]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T06:17:37.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:37 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:37.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:37 smithi196 ceph-mon[30578]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:37 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/3683244918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]: dispatch 2024-09-16T06:17:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:37 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]: dispatch 2024-09-16T06:17:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:37 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd5c0769-e97a-483f-94a3-73b9df643a92"}]': finished 2024-09-16T06:17:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:37 smithi196 ceph-mon[30578]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T06:17:37.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:37 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:38.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:38 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/2505580926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:17:38.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:38 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/2505580926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:17:38.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:38 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/2505580926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:17:39.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:39 smithi196 ceph-mon[30578]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:39.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:39 smithi190 ceph-mon[25888]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:39.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:39 smithi190 ceph-mon[39731]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:41.747 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:41 smithi196 ceph-mon[30578]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:41 smithi190 ceph-mon[25888]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:41.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:41 smithi190 ceph-mon[39731]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:43.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:43 smithi196 ceph-mon[30578]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:43.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:43 smithi190 ceph-mon[25888]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:43 smithi190 ceph-mon[39731]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:44.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:44 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T06:17:44.663 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:44 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:44.663 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:44 smithi196 ceph-mon[30578]: Deploying daemon osd.4 on smithi196 2024-09-16T06:17:44.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:44 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T06:17:44.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:44 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:44.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:44 smithi190 ceph-mon[25888]: Deploying daemon osd.4 on smithi196 2024-09-16T06:17:44.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:44 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T06:17:44.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:44 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:44.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:44 smithi190 ceph-mon[39731]: Deploying daemon osd.4 on smithi196 2024-09-16T06:17:45.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:45 smithi196 ceph-mon[30578]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:45.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:45 smithi190 ceph-mon[25888]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:45.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:45 smithi190 ceph-mon[39731]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:47.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:47 smithi196 ceph-mon[30578]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:47.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:47 smithi190 ceph-mon[25888]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:47.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:47 smithi190 ceph-mon[39731]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:49.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:49 smithi196 ceph-mon[30578]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:49.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:49.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:49.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:49 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:49.649 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[25888]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:49.649 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:49.649 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:49.649 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:49.649 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[39731]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:49.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:49.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:49.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:49 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:51 smithi196 ceph-mon[30578]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:51.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:51 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:51 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:51 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:51.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:51 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:51.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:51 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.601 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-09-16T06:17:51.837 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[25888]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:51.837 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.837 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.837 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:51.837 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:51.837 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[39731]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:51.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:51.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:51.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:51.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:51 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:52.195 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T06:17:52.197 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/nvme2n1... 2024-09-16T06:17:52.198 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-16T06:17:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:52 smithi196 ceph-mon[30578]: from='osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T06:17:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:52 smithi196 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T06:17:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:52 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:52 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:52 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:52.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[25888]: from='osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T06:17:52.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[25888]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T06:17:52.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:52.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:52.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:52.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[39731]: from='osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T06:17:52.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[39731]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T06:17:52.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:17:52.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:52.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:52 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:53.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-mon[30578]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:53.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T06:17:53.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-mon[30578]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T06:17:53.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:53.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-mon[30578]: from='osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:17:53.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:17:53.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:17:53.389+0000 7faa4a050640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T06:17:53.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:17:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:17:53.395+0000 7faa45e79640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:17:53.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[25888]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:53.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[25888]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T06:17:53.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[25888]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T06:17:53.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[25888]: from='osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[25888]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[39731]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[39731]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[39731]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[39731]: from='osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:17:53.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:53 smithi190 ceph-mon[39731]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:17:53.879 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790] boot 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T06:17:54.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790] boot 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:17:54.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T06:17:54.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:54.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: osd.4 [v2:172.21.15.196:6800/2723520790,v1:172.21.15.196:6801/2723520790] boot 2024-09-16T06:17:54.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T06:17:54.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:17:55.759 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:55.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:55 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:17:55.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:17:55.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:55.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T06:17:55.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T06:17:55.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:17:55.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:55.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:55.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:17:55.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:55 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:17:56.506 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme2n1 2024-09-16T06:17:56.506 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:17:56.510 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-16T06:17:56.525 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-16T06:17:56.525 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-16T06:17:56.525 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126011 s, 832 MB/s 2024-09-16T06:17:56.525 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-16T06:17:56.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:56 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi196 2024-09-16T06:17:56.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:56 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-16T06:17:56.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:56 smithi196 ceph-mon[30578]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T06:17:56.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:56 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi196 2024-09-16T06:17:56.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:56 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-16T06:17:56.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:56 smithi190 ceph-mon[25888]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T06:17:56.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:56 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi196 2024-09-16T06:17:56.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:56 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-16T06:17:56.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:56 smithi190 ceph-mon[39731]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T06:17:57.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme2n1 2024-09-16T06:17:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:17:57.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:57 smithi196 ceph-mon[30578]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:17:57.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:57 smithi196 ceph-mon[30578]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T06:17:57.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:57 smithi190 ceph-mon[25888]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:17:57.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:57 smithi190 ceph-mon[25888]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T06:17:57.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:57 smithi190 ceph-mon[39731]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:17:57.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:57 smithi190 ceph-mon[39731]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T06:17:59.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:59 smithi196 ceph-mon[30578]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T06:17:59.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:59 smithi196 ceph-mon[30578]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:17:59.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:59 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:59.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:59 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:59.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:17:59 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[25888]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[25888]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[39731]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[39731]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:17:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:17:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:17:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:17:59 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:00.513 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:00 smithi196 ceph-mon[30578]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:00.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:00 smithi190 ceph-mon[25888]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:00.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:00 smithi190 ceph-mon[39731]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:01.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:01 smithi196 ceph-mon[30578]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:18:01.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:01 smithi190 ceph-mon[25888]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:18:01.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:01 smithi190 ceph-mon[39731]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:18:02.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:02 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/4070772678' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]: dispatch 2024-09-16T06:18:02.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:02 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]: dispatch 2024-09-16T06:18:02.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:02 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]': finished 2024-09-16T06:18:02.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:02 smithi196 ceph-mon[30578]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T06:18:02.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:02 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/4070772678' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]: dispatch 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]: dispatch 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]': finished 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[39731]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/4070772678' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]: dispatch 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]: dispatch 2024-09-16T06:18:02.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc"}]': finished 2024-09-16T06:18:02.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[25888]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T06:18:02.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:02 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:03.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:03 smithi190 ceph-mon[39731]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:03.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:03 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/663350271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:03.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:03 smithi190 ceph-mon[25888]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:03.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:03 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/663350271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:03 smithi196 ceph-mon[30578]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:03 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/663350271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:05.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:05 smithi196 ceph-mon[30578]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T06:18:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:05 smithi190 ceph-mon[25888]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T06:18:05.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:05 smithi190 ceph-mon[39731]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T06:18:07.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:07 smithi196 ceph-mon[30578]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:07.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:07 smithi190 ceph-mon[25888]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:07.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:07 smithi190 ceph-mon[39731]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:08.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T06:18:08.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:08.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T06:18:08.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:08.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T06:18:08.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:09.459 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:09 smithi196 ceph-mon[30578]: Deploying daemon osd.5 on smithi196 2024-09-16T06:18:09.459 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:09 smithi196 ceph-mon[30578]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:09.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:09 smithi190 ceph-mon[25888]: Deploying daemon osd.5 on smithi196 2024-09-16T06:18:09.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:09 smithi190 ceph-mon[25888]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:09.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:09 smithi190 ceph-mon[39731]: Deploying daemon osd.5 on smithi196 2024-09-16T06:18:09.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:09 smithi190 ceph-mon[39731]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:11.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:11 smithi196 ceph-mon[30578]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:11.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:11 smithi190 ceph-mon[25888]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:11.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:11 smithi190 ceph-mon[39731]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:13.470 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:13 smithi196 ceph-mon[30578]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:13.470 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:13.470 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:13.470 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:13 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[25888]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[39731]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:13.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:13 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:15 smithi196 ceph-mon[30578]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:15.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:15 smithi190 ceph-mon[25888]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:15.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:15 smithi190 ceph-mon[39731]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:16.509 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-16T06:18:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:16.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:16.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:16.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T06:18:16.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T06:18:16.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:16.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:16.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:16 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:17.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:17.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:17.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T06:18:17.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T06:18:17.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:17.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.099 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:16 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:17.116 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T06:18:17.118 INFO:tasks.cephadm:Deploying osd.6 on smithi196 with /dev/nvme3n1... 2024-09-16T06:18:17.118 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-16T06:18:18.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T06:18:18.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-mon[30578]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T06:18:18.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:18.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-mon[30578]: from='osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:18.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:18.016 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:18:17.710+0000 7f78820ba640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T06:18:18.016 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:18:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:18:17.714+0000 7f787dee3640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:18:18.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[25888]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T06:18:18.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[25888]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T06:18:18.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:18.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[25888]: from='osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:18.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[25888]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:18.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[39731]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T06:18:18.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[39731]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T06:18:18.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:18.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[39731]: from='osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:18.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:17 smithi190 ceph-mon[39731]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:18.341 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:18:18.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:18 smithi196 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:18:18.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:18 smithi196 ceph-mon[30578]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T06:18:18.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:18 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:18.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:18 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:18.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:18 smithi196 ceph-mon[30578]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[25888]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[25888]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[25888]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[39731]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:18:19.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[39731]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T06:18:19.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:18 smithi190 ceph-mon[39731]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T06:18:19.958 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:19 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:18:19.958 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:19 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:18:19.958 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:19 smithi196 ceph-mon[30578]: osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070] boot 2024-09-16T06:18:19.958 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:19 smithi196 ceph-mon[30578]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T06:18:19.958 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:19 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.958 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:19 smithi196 ceph-mon[30578]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T06:18:19.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:18:19.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:18:19.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[39731]: osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070] boot 2024-09-16T06:18:19.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[39731]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[39731]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[25888]: osd.5 [v2:172.21.15.196:6808/561616070,v1:172.21.15.196:6809/561616070] boot 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[25888]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:18:19.999 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:19 smithi190 ceph-mon[25888]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T06:18:20.575 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme3n1 2024-09-16T06:18:20.575 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:18:20.579 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-16T06:18:20.595 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-16T06:18:20.595 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-16T06:18:20.595 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136676 s, 767 MB/s 2024-09-16T06:18:20.595 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-16T06:18:21.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:21 smithi190 ceph-mon[25888]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T06:18:21.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:21 smithi190 ceph-mon[25888]: pgmap v134: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:21.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:21.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:21 smithi190 ceph-mon[39731]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T06:18:21.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:21 smithi190 ceph-mon[39731]: pgmap v134: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:21.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:21.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:21 smithi196 ceph-mon[30578]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T06:18:21.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:21 smithi196 ceph-mon[30578]: pgmap v134: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:21.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:21.681 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme3n1 2024-09-16T06:18:21.874 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:18:22.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi196 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:22.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:22 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi196 2024-09-16T06:18:22.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:22.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:22.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-16T06:18:22.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T06:18:22.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:22.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:22.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi196 2024-09-16T06:18:22.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:22.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:22.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-16T06:18:22.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T06:18:22.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:22.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:22.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:22.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:22 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:23.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:23 smithi196 ceph-mon[30578]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T06:18:23.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:23 smithi196 ceph-mon[30578]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:23 smithi190 ceph-mon[25888]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T06:18:23.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:23 smithi190 ceph-mon[25888]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:23.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:23 smithi190 ceph-mon[39731]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T06:18:23.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:23 smithi190 ceph-mon[39731]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:24.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:24 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:18:24.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:24 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:18:24.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:24 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:24 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:18:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:24 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:18:24.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:24 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:24.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:24 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:18:24.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:24 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:18:24.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:24 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:25.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:25 smithi196 ceph-mon[30578]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:25.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:25 smithi196 ceph-mon[30578]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:18:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:25 smithi190 ceph-mon[25888]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:25 smithi190 ceph-mon[25888]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:18:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:25 smithi190 ceph-mon[39731]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:25 smithi190 ceph-mon[39731]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:18:26.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:26 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/3009542112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]: dispatch 2024-09-16T06:18:26.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:26 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]: dispatch 2024-09-16T06:18:26.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:26 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]': finished 2024-09-16T06:18:26.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:26 smithi196 ceph-mon[30578]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T06:18:26.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:26 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:26.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/3009542112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]: dispatch 2024-09-16T06:18:26.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]: dispatch 2024-09-16T06:18:26.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]': finished 2024-09-16T06:18:26.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[25888]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T06:18:26.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:26.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/3009542112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]: dispatch 2024-09-16T06:18:26.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]: dispatch 2024-09-16T06:18:26.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac660cff-3f86-4a98-8cbc-97478337878b"}]': finished 2024-09-16T06:18:26.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[39731]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T06:18:26.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:26 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:27.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:27 smithi190 ceph-mon[25888]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:27.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:27 smithi190 ceph-mon[39731]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:27.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:27 smithi196 ceph-mon[30578]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:28.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:28 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/4091533656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:28.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:28 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/4091533656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:28.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/4091533656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:29.549 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:29 smithi196 ceph-mon[30578]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T06:18:29.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:29 smithi190 ceph-mon[25888]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T06:18:29.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:29 smithi190 ceph-mon[39731]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T06:18:31.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:31 smithi196 ceph-mon[30578]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:31.692 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:31 smithi190 ceph-mon[25888]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:31.692 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:31 smithi190 ceph-mon[39731]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:33.736 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:33 smithi196 ceph-mon[30578]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:33.736 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T06:18:33.736 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:33.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:33 smithi190 ceph-mon[25888]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:33.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T06:18:33.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:33.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:33 smithi190 ceph-mon[39731]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:33.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:33 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T06:18:33.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:33 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:34 smithi196 ceph-mon[30578]: Deploying daemon osd.6 on smithi196 2024-09-16T06:18:34.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:34 smithi190 ceph-mon[25888]: Deploying daemon osd.6 on smithi196 2024-09-16T06:18:34.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:34 smithi190 ceph-mon[39731]: Deploying daemon osd.6 on smithi196 2024-09-16T06:18:35.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:35 smithi196 ceph-mon[30578]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:35.846 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:35 smithi190 ceph-mon[25888]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:35.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:35 smithi190 ceph-mon[39731]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:37.755 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:37 smithi196 ceph-mon[30578]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:37.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:37 smithi190 ceph-mon[25888]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:37.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:37 smithi190 ceph-mon[39731]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:38.631 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:38 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:38.631 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:38 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:38.631 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:38 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:38.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:38 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:38.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:38 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:38.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:38 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:38.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:38 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:38.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:38 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:38.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:38 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:39.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:39 smithi196 ceph-mon[30578]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:39.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:39 smithi190 ceph-mon[25888]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:39.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:39 smithi190 ceph-mon[39731]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:41.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T06:18:41.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:41 smithi196 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T06:18:41.554 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 6 on host 'smithi196' 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T06:18:41.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[25888]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T06:18:41.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:41 smithi190 ceph-mon[39731]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T06:18:42.165 DEBUG:teuthology.orchestra.run.smithi196:osd.6> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T06:18:42.168 INFO:tasks.cephadm:Deploying osd.7 on smithi196 with /dev/nvme4n1... 2024-09-16T06:18:42.168 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-16T06:18:42.450 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:42.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:42 smithi196 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:42.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:42.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:42.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[25888]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T06:18:42.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T06:18:42.850 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:42.850 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:42.850 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:42 smithi190 ceph-mon[39731]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:18:42.941 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:18:43.378 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:18:43.124+0000 7f0741599640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T06:18:43.379 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:18:43.130+0000 7f07393ba640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:18:43.717 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-mon[30578]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:43.717 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:18:43.717 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-mon[30578]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T06:18:43.717 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:43.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:43.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:43 smithi196 ceph-mon[30578]: from='osd.6 ' entity='osd.6' 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[39731]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[39731]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[39731]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[39731]: from='osd.6 ' entity='osd.6' 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[25888]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[25888]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:18:43.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[25888]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T06:18:43.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:43.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:43.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:43 smithi190 ceph-mon[25888]: from='osd.6 ' entity='osd.6' 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894] boot 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T06:18:44.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:44 smithi196 ceph-mon[30578]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894] boot 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[25888]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:18:45.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: osd.6 [v2:172.21.15.196:6816/3795627894,v1:172.21.15.196:6817/3795627894] boot 2024-09-16T06:18:45.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T06:18:45.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:18:45.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T06:18:45.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:44 smithi190 ceph-mon[39731]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:46.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:46 smithi196 ceph-mon[30578]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T06:18:46.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:46 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:46.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:46 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:46.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:46 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:46 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:46 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[25888]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T06:18:46.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:46.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:46.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[39731]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T06:18:46.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:46.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:46.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:46 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:18:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme4n1 2024-09-16T06:18:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T06:18:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-16T06:18:46.714 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-16T06:18:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-16T06:18:46.714 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127997 s, 819 MB/s 2024-09-16T06:18:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-16T06:18:47.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi196 2024-09-16T06:18:47.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-16T06:18:47.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:47.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:47.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T06:18:47.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:47.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:47.345 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:47 smithi196 ceph-mon[30578]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:47.366 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme4n1 2024-09-16T06:18:47.558 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:18:47.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi196 2024-09-16T06:18:47.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-16T06:18:47.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:47.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:47.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[25888]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi196 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:18:47.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:47 smithi190 ceph-mon[39731]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:48.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:48 smithi196 ceph-mon[30578]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T06:18:48.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:48 smithi190 ceph-mon[25888]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T06:18:48.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:48 smithi190 ceph-mon[39731]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T06:18:49.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:49 smithi196 ceph-mon[30578]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:49.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:49 smithi190 ceph-mon[25888]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:49.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:49 smithi190 ceph-mon[39731]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 589 MiB used, 625 GiB / 626 GiB avail 2024-09-16T06:18:50.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:50 smithi196 ceph-mon[30578]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:50.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:18:50.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:18:50.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:50 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:50.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[25888]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:50.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:18:50.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:18:50.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:50.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[39731]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:18:50.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T06:18:50.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T06:18:50.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:50 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:51.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:51 smithi190 ceph-mon[25888]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:51.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:51 smithi190 ceph-mon[39731]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:51.710 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:51 smithi196 ceph-mon[30578]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/3569139838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]: dispatch 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]: dispatch 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[25888]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]': finished 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[25888]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/3569139838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]: dispatch 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]: dispatch 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[39731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]': finished 2024-09-16T06:18:52.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[39731]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T06:18:52.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:52 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:18:52.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:52 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/3569139838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]: dispatch 2024-09-16T06:18:52.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:52 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]: dispatch 2024-09-16T06:18:52.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:52 smithi196 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07420cdf-64f6-4c03-878e-9b31d6b466e5"}]': finished 2024-09-16T06:18:52.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:52 smithi196 ceph-mon[30578]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T06:18:52.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:52 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:18:53.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:53 smithi190 ceph-mon[25888]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T06:18:53.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:53 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/3520918746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:53.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:53 smithi190 ceph-mon[39731]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T06:18:53.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:53 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/3520918746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:53.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:53 smithi196 ceph-mon[30578]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T06:18:53.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:53 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/3520918746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T06:18:55.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:55 smithi196 ceph-mon[30578]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:55.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:55 smithi190 ceph-mon[25888]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:55.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:55 smithi190 ceph-mon[39731]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:18:57.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:57 smithi196 ceph-mon[30578]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T06:18:57.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:57 smithi190 ceph-mon[25888]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T06:18:57.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:57 smithi190 ceph-mon[39731]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T06:18:58.709 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:58 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T06:18:58.709 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:58 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:58.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:58 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T06:18:58.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:58 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:58 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T06:18:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:58 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:18:59.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:59 smithi196 ceph-mon[30578]: Deploying daemon osd.7 on smithi196 2024-09-16T06:18:59.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:18:59 smithi196 ceph-mon[30578]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:59.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:59 smithi190 ceph-mon[25888]: Deploying daemon osd.7 on smithi196 2024-09-16T06:18:59.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:18:59 smithi190 ceph-mon[25888]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:18:59.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:59 smithi190 ceph-mon[39731]: Deploying daemon osd.7 on smithi196 2024-09-16T06:18:59.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:18:59 smithi190 ceph-mon[39731]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T06:19:01.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:01 smithi196 ceph-mon[30578]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:01.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:01 smithi190 ceph-mon[25888]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:01 smithi190 ceph-mon[39731]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:03 smithi196 ceph-mon[30578]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:03 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:03 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:03 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[25888]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[39731]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:03.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:03 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:05.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:05 smithi196 ceph-mon[30578]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:05 smithi190 ceph-mon[25888]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:05.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:05 smithi190 ceph-mon[39731]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:06.430 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 7 on host 'smithi196' 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:06.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:06.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:06 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.066 DEBUG:teuthology.orchestra.run.smithi196:osd.7> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T06:19:07.068 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-16T06:19:07.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:06 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:07.278 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:08.063 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T06:19:08.064 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-mon[30578]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T06:19:08.064 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.064 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-mon[30578]: from='osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:19:08.064 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:19:08.064 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:19:07.955+0000 7fa62388f640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T06:19:08.064 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:19:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:19:07.963+0000 7fa61f6b8640 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:19:08.079 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[25888]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[25888]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[25888]: from='osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[25888]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[39731]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[39731]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[39731]: from='osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:19:08.080 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:07 smithi190 ceph-mon[39731]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076] boot 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T06:19:08.976 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076] boot 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T06:19:08.977 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:08 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:09.129 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:19:09.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-16T06:19:09.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T06:19:09.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:09.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:09.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T06:19:09.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: osd.7 [v2:172.21.15.196:6824/2986179076,v1:172.21.15.196:6825/2986179076] boot 2024-09-16T06:19:09.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T06:19:09.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:08 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:09.707 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726467548,"num_in_osds":8,"osd_in_since":1726467531,"num_remapped_pgs":1} 2024-09-16T06:19:09.708 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T06:19:09.941 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:10.170 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:09 smithi196 ceph-mon[30578]: purged_snaps scrub starts 2024-09-16T06:19:10.170 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:09 smithi196 ceph-mon[30578]: purged_snaps scrub ok 2024-09-16T06:19:10.170 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:09 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/3251556669' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T06:19:10.170 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:09 smithi196 ceph-mon[30578]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T06:19:10.234 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[25888]: purged_snaps scrub starts 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[25888]: purged_snaps scrub ok 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3251556669' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[25888]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[39731]: purged_snaps scrub starts 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[39731]: purged_snaps scrub ok 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/3251556669' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T06:19:10.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:09 smithi190 ceph-mon[39731]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T06:19:11.469 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: Detected new or changed devices on smithi196 2024-09-16T06:19:11.469 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.469 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.469 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.469 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.469 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[25888]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: Detected new or changed devices on smithi196 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.470 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T06:19:11.471 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:11 smithi190 ceph-mon[39731]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:11.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: Detected new or changed devices on smithi196 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:11.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:11.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:11.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T06:19:11.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:11 smithi196 ceph-mon[30578]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:11.863 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:19:11.863 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":58,"fsid":"bd41205c-73f2-11ef-bceb-c7b262605968","created":"2024-09-16T06:14:07.938544+0000","modified":"2024-09-16T06:19:11.245369+0000","last_up_change":"2024-09-16T06:19:08.945843+0000","last_in_change":"2024-09-16T06:18:51.658721+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:17:08.572382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0a4292e-21dc-43b1-9cfa-0d5de57a2484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6803","nonce":62621509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6805","nonce":62621509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6809","nonce":62621509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6807","nonce":62621509}]},"public_addr":"172.21.15.190:6803/62621509","cluster_addr":"172.21.15.190:6805/62621509","heartbeat_back_addr":"172.21.15.190:6809/62621509","heartbeat_front_addr":"172.21.15.190:6807/62621509","state":["exists","up"]},{"osd":1,"uuid":"52400e70-bb15-46cf-9280-8b9c44a9c16b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6811","nonce":525777266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6813","nonce":525777266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6817","nonce":525777266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6815","nonce":525777266}]},"public_addr":"172.21.15.190:6811/525777266","cluster_addr":"172.21.15.190:6813/525777266","heartbeat_back_addr":"172.21.15.190:6817/525777266","heartbeat_front_addr":"172.21.15.190:6815/525777266","state":["exists","up"]},{"osd":2,"uuid":"46b8a2a4-b143-4784-8780-0ed28f8e0276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6819","nonce":3137551404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6821","nonce":3137551404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6825","nonce":3137551404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6823","nonce":3137551404}]},"public_addr":"172.21.15.190:6819/3137551404","cluster_addr":"172.21.15.190:6821/3137551404","heartbeat_back_addr":"172.21.15.190:6825/3137551404","heartbeat_front_addr":"172.21.15.190:6823/3137551404","state":["exists","up"]},{"osd":3,"uuid":"3cbdf234-9282-4ab1-a636-03abcecece3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6827","nonce":1507636168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6829","nonce":1507636168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6833","nonce":1507636168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6831","nonce":1507636168}]},"public_addr":"172.21.15.190:6827/1507636168","cluster_addr":"172.21.15.190:6829/1507636168","heartbeat_back_addr":"172.21.15.190:6833/1507636168","heartbeat_front_addr":"172.21.15.190:6831/1507636168","state":["exists","up"]},{"osd":4,"uuid":"fd5c0769-e97a-483f-94a3-73b9df643a92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6801","nonce":2723520790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6803","nonce":2723520790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6807","nonce":2723520790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6805","nonce":2723520790}]},"public_addr":"172.21.15.196:6801/2723520790","cluster_addr":"172.21.15.196:6803/2723520790","heartbeat_back_addr":"172.21.15.196:6807/2723520790","heartbeat_front_addr":"172.21.15.196:6805/2723520790","state":["exists","up"]},{"osd":5,"uuid":"c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6809","nonce":561616070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6811","nonce":561616070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6815","nonce":561616070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6813","nonce":561616070}]},"public_addr":"172.21.15.196:6809/561616070","cluster_addr":"172.21.15.196:6811/561616070","heartbeat_back_addr":"172.21.15.196:6815/561616070","heartbeat_front_addr":"172.21.15.196:6813/561616070","state":["exists","up"]},{"osd":6,"uuid":"ac660cff-3f86-4a98-8cbc-97478337878b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6817","nonce":3795627894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6819","nonce":3795627894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6823","nonce":3795627894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6821","nonce":3795627894}]},"public_addr":"172.21.15.196:6817/3795627894","cluster_addr":"172.21.15.196:6819/3795627894","heartbeat_back_addr":"172.21.15.196:6823/3795627894","heartbeat_front_addr":"172.21.15.196:6821/3795627894","state":["exists","up"]},{"osd":7,"uuid":"07420cdf-64f6-4c03-878e-9b31d6b466e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6825","nonce":2986179076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6827","nonce":2986179076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6831","nonce":2986179076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6829","nonce":2986179076}]},"public_addr":"172.21.15.196:6825/2986179076","cluster_addr":"172.21.15.196:6827/2986179076","heartbeat_back_addr":"172.21.15.196:6831/2986179076","heartbeat_front_addr":"172.21.15.196:6829/2986179076","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:16:16.262056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:16:40.843599+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:05.820305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:30.199586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:52.476721+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:18:17.390373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:18:42.103890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:19:07.208019+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:0/522299671":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6801/2541445906":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/3600548341":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/3341383511":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/2985493186":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/3599123797":"2024-09-17T06:14:31.152761+0000","172.21.15.190:6800/2541445906":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6801/1748866605":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/4051857884":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6800/1748866605":"2024-09-17T06:14:31.152761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:19:12.492 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:12 smithi190 ceph-mon[25888]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T06:19:12.493 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:12 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3314386415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:19:12.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:12 smithi190 ceph-mon[39731]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T06:19:12.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:12 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/3314386415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:19:12.513 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T06:17:08.572382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T06:19:12.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T06:19:12.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:12 smithi196 ceph-mon[30578]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T06:19:12.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:12 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/3314386415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:19:12.705 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:13.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:13 smithi196 ceph-mon[30578]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T06:19:13.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:13 smithi196 ceph-mon[30578]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:13.524 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:13 smithi190 ceph-mon[25888]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T06:19:13.524 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:13 smithi190 ceph-mon[25888]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:13.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:13 smithi190 ceph-mon[39731]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T06:19:13.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:13 smithi190 ceph-mon[39731]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:14.450 INFO:teuthology.orchestra.run.smithi190.stdout:pg_num: 1 2024-09-16T06:19:15.188 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi190 2024-09-16T06:19:15.188 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi190=foo.a' 2024-09-16T06:19:15.394 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:15.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:15 smithi196 ceph-mon[30578]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:19:15.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:15 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/588913325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T06:19:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:15 smithi190 ceph-mon[25888]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:19:15.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:15 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/588913325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T06:19:15.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:15 smithi190 ceph-mon[39731]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T06:19:15.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:15 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/588913325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T06:19:17.153 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled rgw.foo.a update... 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T06:19:17.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.466 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:17 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:17.639 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:17 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:17.744 DEBUG:teuthology.orchestra.run.smithi190:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-16T06:19:17.747 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi196 2024-09-16T06:19:17.747 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-16T06:19:17.939 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:18.536 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:18 smithi190 ceph-mon[39731]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi190=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:18.536 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:18 smithi190 ceph-mon[39731]: Saving service rgw.foo.a spec with placement smithi190=foo.a;count:1 2024-09-16T06:19:18.536 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:18 smithi190 ceph-mon[39731]: Deploying daemon rgw.foo.a on smithi190 2024-09-16T06:19:18.537 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:18 smithi190 ceph-mon[25888]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi190=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:18.537 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:18 smithi190 ceph-mon[25888]: Saving service rgw.foo.a spec with placement smithi190=foo.a;count:1 2024-09-16T06:19:18.537 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:18 smithi190 ceph-mon[25888]: Deploying daemon rgw.foo.a on smithi190 2024-09-16T06:19:18.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:18 smithi196 ceph-mon[30578]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi190=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:18.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:18 smithi196 ceph-mon[30578]: Saving service rgw.foo.a spec with placement smithi190=foo.a;count:1 2024-09-16T06:19:18.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:18 smithi196 ceph-mon[30578]: Deploying daemon rgw.foo.a on smithi190 2024-09-16T06:19:19.097 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 systemd[1]: Starting Ceph rgw.foo.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:19:19.373 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 podman[77588]: 2024-09-16 06:19:19.138762056 +0000 UTC m=+0.015674153 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:19:19.374 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 podman[77588]: 2024-09-16 06:19:19.27199869 +0000 UTC m=+0.148910815 container create 9bb2fcabd973e1b6c89bbff9531f854ce0b88d87b348b4c5bd91dd598b3c1b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T06:19:19.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:19 smithi196 ceph-mon[30578]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:19:19.846 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:19 smithi190 ceph-mon[25888]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:19:19.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:19 smithi190 ceph-mon[39731]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T06:19:19.847 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 podman[77588]: 2024-09-16 06:19:19.500067688 +0000 UTC m=+0.376979775 container init 9bb2fcabd973e1b6c89bbff9531f854ce0b88d87b348b4c5bd91dd598b3c1b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T06:19:19.847 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 podman[77588]: 2024-09-16 06:19:19.505613631 +0000 UTC m=+0.382525714 container start 9bb2fcabd973e1b6c89bbff9531f854ce0b88d87b348b4c5bd91dd598b3c1b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T06:19:19.847 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 bash[77588]: 9bb2fcabd973e1b6c89bbff9531f854ce0b88d87b348b4c5bd91dd598b3c1b18 2024-09-16T06:19:19.847 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:19:19 smithi190 systemd[1]: Started Ceph rgw.foo.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:19:20.387 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'datapool' created 2024-09-16T06:19:20.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/3964711100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/3964711100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T06:19:20.644 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T06:19:20.645 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.645 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.645 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.645 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.645 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.645 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:20 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:20.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/3964711100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T06:19:20.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T06:19:20.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:20.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:20 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:21.130 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-16T06:19:21.329 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: Saving service rgw.foo.a spec with placement smithi190=foo.a;count:1 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:21.414 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: Saving service rgw.foo.a spec with placement smithi190=foo.a;count:1 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:21.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:21 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:21.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: Saving service rgw.foo.a spec with placement smithi190=foo.a;count:1 2024-09-16T06:19:21.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T06:19:21.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T06:19:21.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T06:19:21.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T06:19:21.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T06:19:21.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:21.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:21.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:21.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:21 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:22.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: Checking dashboard <-> RGW credentials 2024-09-16T06:19:22.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T06:19:22.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T06:19:22.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T06:19:22.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:22 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: Checking dashboard <-> RGW credentials 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: Checking dashboard <-> RGW credentials 2024-09-16T06:19:22.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T06:19:22.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T06:19:22.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T06:19:22.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:22.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:22 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T06:19:23.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: pgmap v185: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:23.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T06:19:23.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/4124951833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T06:19:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T06:19:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T06:19:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T06:19:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T06:19:23.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:23 smithi196 ceph-mon[30578]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: pgmap v185: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/4124951833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T06:19:23.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[25888]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: pgmap v185: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/4124951833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T06:19:23.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:23 smithi190 ceph-mon[39731]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: pgmap v188: 68 pgs: 8 creating+peering, 10 unknown, 50 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[39731]: Cluster is now healthy 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: pgmap v188: 68 pgs: 8 creating+peering, 10 unknown, 50 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T06:19:25.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:25.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T06:19:25.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T06:19:25.598 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:25 smithi190 ceph-mon[25888]: Cluster is now healthy 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: pgmap v188: 68 pgs: 8 creating+peering, 10 unknown, 50 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/27618125' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T06:19:25.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:25 smithi196 ceph-mon[30578]: Cluster is now healthy 2024-09-16T06:19:26.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.196 --placement '1;smithi196=iscsi.a' 2024-09-16T06:19:26.267 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:26.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:26 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T06:19:26.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:26 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T06:19:26.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:26 smithi196 ceph-mon[30578]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T06:19:26.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:26 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T06:19:26.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:26 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T06:19:26.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:26 smithi190 ceph-mon[25888]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T06:19:26.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:26 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/774663609' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T06:19:26.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:26 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T06:19:26.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:26 smithi190 ceph-mon[39731]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T06:19:27.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:27 smithi196 ceph-mon[30578]: pgmap v191: 100 pgs: 12 creating+peering, 28 unknown, 60 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T06:19:27.690 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:27 smithi196 ceph-mon[30578]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T06:19:27.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:27 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/538652057' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:27 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.691 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:27 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[25888]: pgmap v191: 100 pgs: 12 creating+peering, 28 unknown, 60 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T06:19:27.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[25888]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T06:19:27.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/538652057' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[39731]: pgmap v191: 100 pgs: 12 creating+peering, 28 unknown, 60 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-16T06:19:27.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[39731]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T06:19:27.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/538652057' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:27.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:27 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T06:19:28.075 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled iscsi.datapool update... 2024-09-16T06:19:28.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T06:19:28.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T06:19:28.729 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/538652057' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T06:19:28.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:28 smithi196 ceph-mon[30578]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T06:19:28.761 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-16T06:19:28.761 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:19:28.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/538652057' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T06:19:28.786 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[25888]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/538652057' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2557367005' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T06:19:28.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:28 smithi190 ceph-mon[39731]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T06:19:28.790 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:19:28.790 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T06:19:28.818 DEBUG:teuthology.orchestra.run.smithi196:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-16T06:19:28.862 INFO:tasks.cephadm:Adding prometheus.a on smithi196 2024-09-16T06:19:28.862 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi196=a' 2024-09-16T06:19:29.524 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-16T06:19:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: pgmap v194: 132 pgs: 11 creating+peering, 42 unknown, 79 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 1 op/s 2024-09-16T06:19:29.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T06:19:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T06:19:29.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:29 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: pgmap v194: 132 pgs: 11 creating+peering, 42 unknown, 79 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 1 op/s 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T06:19:29.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: pgmap v194: 132 pgs: 11 creating+peering, 42 unknown, 79 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 1 op/s 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T06:19:29.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:29 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:30.479 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:30 smithi196 ceph-mon[30578]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-16T06:19:30.846 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:30 smithi190 ceph-mon[39731]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-16T06:19:30.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:30 smithi190 ceph-mon[25888]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-16T06:19:31.686 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:31 smithi196 ceph-mon[30578]: pgmap v196: 132 pgs: 7 creating+peering, 125 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 4.5 KiB/s wr, 126 op/s 2024-09-16T06:19:31.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:31 smithi196 podman[67559]: 2024-09-16 06:19:31.571342332 +0000 UTC m=+0.322886027 container create cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T06:19:31.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:31 smithi190 ceph-mon[25888]: pgmap v196: 132 pgs: 7 creating+peering, 125 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 4.5 KiB/s wr, 126 op/s 2024-09-16T06:19:31.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:31 smithi190 ceph-mon[39731]: pgmap v196: 132 pgs: 7 creating+peering, 125 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 4.5 KiB/s wr, 126 op/s 2024-09-16T06:19:32.113 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled prometheus update... 2024-09-16T06:19:32.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:32 smithi196 podman[67559]: 2024-09-16 06:19:32.082819467 +0000 UTC m=+0.834363169 container init cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:19:32.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:32 smithi196 podman[67559]: 2024-09-16 06:19:32.08784033 +0000 UTC m=+0.839384035 container start cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T06:19:32.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:32 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu[67616]: log file path now is '/var/log/tcmu-runner.log' 2024-09-16T06:19:32.473 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:32 smithi196 bash[67559]: cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a 2024-09-16T06:19:32.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:32 smithi196 podman[67712]: 2024-09-16 06:19:32.375954525 +0000 UTC m=+0.146351429 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:19:32.752 DEBUG:teuthology.orchestra.run.smithi196:prometheus.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-16T06:19:32.754 INFO:tasks.cephadm:Adding node-exporter.a on smithi190 2024-09-16T06:19:32.754 INFO:tasks.cephadm:Adding node-exporter.b on smithi196 2024-09-16T06:19:32.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi190=a;smithi196=b' 2024-09-16T06:19:33.015 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:32 smithi196 podman[67712]: 2024-09-16 06:19:32.912615778 +0000 UTC m=+0.683012664 container create cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:19:33.257 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:33.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-mon[30578]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:33.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-mon[30578]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-16T06:19:33.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:33.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-mon[30578]: pgmap v197: 132 pgs: 7 creating+peering, 125 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 104 op/s 2024-09-16T06:19:33.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 podman[67712]: 2024-09-16 06:19:33.098091027 +0000 UTC m=+0.868487930 container init cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T06:19:33.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 podman[67712]: 2024-09-16 06:19:33.10338982 +0000 UTC m=+0.873786709 container start cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T06:19:33.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 bash[67712]: cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 2024-09-16T06:19:33.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 systemd[1]: Started Ceph iscsi.iscsi.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:19:33.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[25888]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:33.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[25888]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-16T06:19:33.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:33.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[25888]: pgmap v197: 132 pgs: 7 creating+peering, 125 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 104 op/s 2024-09-16T06:19:33.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[39731]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:33.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[39731]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-16T06:19:33.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:33.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:33 smithi190 ceph-mon[39731]: pgmap v197: 132 pgs: 7 creating+peering, 125 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 104 op/s 2024-09-16T06:19:33.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug Started the configuration object watcher 2024-09-16T06:19:33.643 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug Checking for config object changes every 1s 2024-09-16T06:19:33.643 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug Processing osd blocklist entries for this node 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug Reading the configuration object to update local LIO configuration 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug Configuration does not have an entry for this host(smithi196.front.sepia.ceph.com) - nothing to define to LIO 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: * Environment: production 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: Use a production WSGI server instead. 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: * Debug mode: off 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug * Running on all addresses. 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: * Running on all addresses. 2024-09-16T06:19:34.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T06:19:34.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug * Running on http://[fe80::ec4:7aff:fe88:7f47]:5000/ (Press CTRL+C to quit) 2024-09-16T06:19:34.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: * Running on http://[fe80::ec4:7aff:fe88:7f47]:5000/ (Press CTRL+C to quit) 2024-09-16T06:19:34.387 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/2829531975' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T06:19:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:34 smithi196 ceph-mon[30578]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/2829531975' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[25888]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T06:19:34.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/2829531975' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T06:19:34.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:34 smithi190 ceph-mon[39731]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T06:19:35.080 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled node-exporter update... 2024-09-16T06:19:35.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:35 smithi190 ceph-mon[25888]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.6 KiB/s wr, 164 op/s 2024-09-16T06:19:35.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:35 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:35.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:35 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:35.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:35 smithi190 ceph-mon[39731]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.6 KiB/s wr, 164 op/s 2024-09-16T06:19:35.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:35 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:35.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:35 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:35.631 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:35 smithi196 ceph-mon[30578]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s rd, 5.6 KiB/s wr, 164 op/s 2024-09-16T06:19:35.631 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:35 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:35.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:35 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:36.369 DEBUG:teuthology.orchestra.run.smithi190:node-exporter.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-16T06:19:36.372 DEBUG:teuthology.orchestra.run.smithi196:node-exporter.b> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-16T06:19:36.375 INFO:tasks.cephadm:Adding alertmanager.a on smithi190 2024-09-16T06:19:36.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi190=a' 2024-09-16T06:19:36.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:36 smithi190 ceph-mon[25888]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi190=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:36.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:36 smithi190 ceph-mon[25888]: Saving service node-exporter spec with placement smithi190=a;smithi196=b;count:2 2024-09-16T06:19:36.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:36 smithi190 ceph-mon[25888]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T06:19:36.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:36 smithi190 ceph-mon[39731]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi190=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:36.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:36 smithi190 ceph-mon[39731]: Saving service node-exporter spec with placement smithi190=a;smithi196=b;count:2 2024-09-16T06:19:36.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:36 smithi190 ceph-mon[39731]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T06:19:36.632 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:36.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:36 smithi196 ceph-mon[30578]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi190=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:36.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:36 smithi196 ceph-mon[30578]: Saving service node-exporter spec with placement smithi190=a;smithi196=b;count:2 2024-09-16T06:19:36.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:36 smithi196 ceph-mon[30578]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T06:19:37.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:37 smithi190 ceph-mon[25888]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.6 KiB/s wr, 141 op/s 2024-09-16T06:19:37.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:37 smithi190 ceph-mon[39731]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.6 KiB/s wr, 141 op/s 2024-09-16T06:19:37.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:37 smithi196 ceph-mon[30578]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.6 KiB/s wr, 141 op/s 2024-09-16T06:19:39.129 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled alertmanager update... 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 114 op/s 2024-09-16T06:19:39.130 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:39 smithi196 ceph-mon[30578]: Deploying daemon prometheus.a on smithi196 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 114 op/s 2024-09-16T06:19:39.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[25888]: Deploying daemon prometheus.a on smithi196 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 114 op/s 2024-09-16T06:19:39.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:39 smithi190 ceph-mon[39731]: Deploying daemon prometheus.a on smithi196 2024-09-16T06:19:39.836 DEBUG:teuthology.orchestra.run.smithi190:alertmanager.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-16T06:19:39.838 INFO:tasks.cephadm:Adding grafana.a on smithi196 2024-09-16T06:19:39.839 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi196=a' 2024-09-16T06:19:40.037 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:40 smithi196 ceph-mon[30578]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:40 smithi196 ceph-mon[30578]: Saving service alertmanager spec with placement smithi190=a;count:1 2024-09-16T06:19:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:40 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:40 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[25888]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[25888]: Saving service alertmanager spec with placement smithi190=a;count:1 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[39731]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[39731]: Saving service alertmanager spec with placement smithi190=a;count:1 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:40.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:40 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:41.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:41 smithi196 ceph-mon[30578]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 51 op/s 2024-09-16T06:19:41.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:41 smithi190 ceph-mon[25888]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 51 op/s 2024-09-16T06:19:41.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:41 smithi190 ceph-mon[39731]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 51 op/s 2024-09-16T06:19:42.612 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled grafana update... 2024-09-16T06:19:43.412 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:43 smithi196 ceph-mon[30578]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 51 op/s 2024-09-16T06:19:43.412 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:43 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:43.577 DEBUG:teuthology.orchestra.run.smithi196:grafana.a> sudo journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@grafana.a.service 2024-09-16T06:19:43.580 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T06:19:43.580 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T06:19:43.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:19:43.786 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:43.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:43 smithi190 ceph-mon[25888]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 51 op/s 2024-09-16T06:19:43.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:43 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:43 smithi190 ceph-mon[39731]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 51 op/s 2024-09-16T06:19:43.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:43 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:44.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:44 smithi190 ceph-mon[25888]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:44.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:44 smithi190 ceph-mon[25888]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-16T06:19:44.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:44 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:19:44.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:44 smithi190 ceph-mon[39731]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:44.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:44 smithi190 ceph-mon[39731]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-16T06:19:44.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:44 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:19:44.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:44 smithi196 ceph-mon[30578]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:19:44.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:44 smithi196 ceph-mon[30578]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-16T06:19:44.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:44 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:19:45.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:45 smithi190 ceph-mon[25888]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-09-16T06:19:45.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:45 smithi190 ceph-mon[39731]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-09-16T06:19:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:45 smithi196 ceph-mon[30578]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-09-16T06:19:45.832 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2024-09-16T06:19:45.832 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQABzudmQE1IMRAAF34pbimS8G8Ejwg38mo8wQ== 2024-09-16T06:19:46.513 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:19:46.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T06:19:46.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T06:19:46.551 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T06:19:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:46 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/1959373835' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:46 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:46.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:46 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T06:19:46.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:46 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/1959373835' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:46.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:46 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:46.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:46 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T06:19:46.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:46 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1959373835' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:46.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:46 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:46.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:46 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T06:19:47.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:47 smithi196 ceph-mon[30578]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T06:19:47.605 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T06:19:47.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:47 smithi190 ceph-mon[25888]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T06:19:47.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:47 smithi190 ceph-mon[39731]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T06:19:49.649 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:49 smithi190 ceph-mon[25888]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T06:19:49.650 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:49 smithi190 ceph-mon[39731]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T06:19:49.714 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:49 smithi196 ceph-mon[30578]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-16T06:19:51.327 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-09-16T06:19:51.327 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQAHzudmkGQ+ExAAz2z9PQ09jIBcA4NQPt4Zsg== 2024-09-16T06:19:51.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:51 smithi196 ceph-mon[30578]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:19:51.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:51 smithi196 ceph-mon[30578]: from='client.? 172.21.15.196:0/3196418012' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:51.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:51 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:51.635 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:51 smithi196 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T06:19:51.821 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[25888]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[25888]: from='client.? 172.21.15.196:0/3196418012' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[25888]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[39731]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[39731]: from='client.? 172.21.15.196:0/3196418012' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T06:19:51.822 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:51 smithi190 ceph-mon[39731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T06:19:52.026 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:19:52.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T06:19:52.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T06:19:52.069 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T06:19:52.069 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T06:19:52.069 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T06:19:52.261 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 podman[69687]: 2024-09-16 06:19:52.112836473 +0000 UTC m=+0.017461916 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T06:19:52.261 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 podman[69687]: 2024-09-16 06:19:52.261157541 +0000 UTC m=+0.165782973 container create e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:19:52.264 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:52.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 podman[69687]: 2024-09-16 06:19:52.472083128 +0000 UTC m=+0.376708567 container init e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:19:52.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 podman[69687]: 2024-09-16 06:19:52.476909191 +0000 UTC m=+0.381534622 container start e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:19:52.517 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.501Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T06:19:52.517 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.501Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T06:19:52.517 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.501Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi196 (none))" 2024-09-16T06:19:52.517 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.501Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T06:19:52.517 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.501Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T06:19:52.517 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.503Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T06:19:52.518 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.503Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T06:19:52.518 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.504Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T06:19:52.518 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.504Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T06:19:52.518 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.505Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T06:19:52.518 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.505Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.919µs 2024-09-16T06:19:52.518 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.505Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T06:19:52.519 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.506Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-16T06:19:52.519 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.506Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=23.64µs wal_replay_duration=489.054µs wbl_replay_duration=173ns total_replay_duration=540.278µs 2024-09-16T06:19:52.519 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.507Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T06:19:52.519 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.507Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T06:19:52.519 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.507Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T06:19:53.015 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.528Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.899851ms db_storage=679ns remote_storage=980ns web_handler=308ns query_engine=561ns scrape=395.244µs scrape_sd=76.613µs notify=987ns notify_sd=2.515µs rules=20.147372ms tracing=5.811µs 2024-09-16T06:19:53.015 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.528Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T06:19:53.015 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:19:52.528Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T06:19:53.016 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 bash[69687]: e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 2024-09-16T06:19:53.016 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:19:52 smithi196 systemd[1]: Started Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:19:53.656 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[25888]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:19:53.657 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.657 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.657 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.657 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T06:19:53.657 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[39731]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:19:53.658 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.658 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.658 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.658 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T06:19:53.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-mon[30578]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:19:53.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' 2024-09-16T06:19:53.685 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T06:19:53.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:19:53.988 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ignoring --setuser ceph since I am not root 2024-09-16T06:19:53.988 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ignoring --setgroup ceph since I am not root 2024-09-16T06:19:53.988 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:53.797+0000 7fe51faec040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:19:53.988 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:53 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:53.859+0000 7fe51faec040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:19:54.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: ignoring --setuser ceph since I am not root 2024-09-16T06:19:54.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: ignoring --setgroup ceph since I am not root 2024-09-16T06:19:54.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-mgr[32680]: -- 172.21.15.196:0/2701645323 <== mon.0 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561d36e2cd00 con 0x561d36e00800 2024-09-16T06:19:54.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:53.799+0000 7f918969e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:19:54.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:53.855+0000 7f918969e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:19:54.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.000+0000 7f918969e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:19:54.265 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.066+0000 7f918969e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:19:54.265 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.209+0000 7f918969e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:19:54.294 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:19:54.294 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.008+0000 7fe51faec040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:19:54.294 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.077+0000 7fe51faec040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:19:54.295 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.224+0000 7fe51faec040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:19:54.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.375+0000 7fe51faec040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:19:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:19:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[25888]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T06:19:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[25888]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-16T06:19:54.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1723714951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T06:19:54.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:19:54.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[39731]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T06:19:54.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[39731]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-16T06:19:54.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/1723714951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T06:19:54.651 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.348+0000 7f918969e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:19:54.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:19:54.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-mon[30578]: from='mgr.14150 172.21.15.190:0/921000726' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T06:19:54.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-mon[30578]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-16T06:19:54.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/1723714951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T06:19:54.925 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.688+0000 7fe51faec040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:19:54.925 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.747+0000 7fe51faec040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:19:54.947 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2827383855},{"type":"v1","addr":"172.21.15.190:6801","nonce":2827383855}]},"active_addr":"172.21.15.190:6801/2827383855","active_change":"2024-09-16T06:14:54.263968+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.190:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3271532069}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2924209252}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3385091135}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2689243789}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":1048924925}]}]} 2024-09-16T06:19:54.951 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T06:19:54.951 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T06:19:54.951 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T06:19:55.015 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.651+0000 7f918969e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:19:55.015 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.710+0000 7f918969e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:19:55.015 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:54 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:54.891+0000 7f918969e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:19:55.150 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:55.271 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:54.930+0000 7fe51faec040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:19:56.015 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:55 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:55.756+0000 7f918969e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:19:56.015 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:55 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:55.822+0000 7f918969e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:19:56.096 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:55.843+0000 7fe51faec040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:19:56.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:55.908+0000 7fe51faec040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:19:56.742 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:56 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:56.628+0000 7fe51faec040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T06:19:56.776 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:56.524+0000 7f918969e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T06:19:56.776 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:56.637+0000 7f918969e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:19:57.026 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:56 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:56.746+0000 7fe51faec040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:19:57.027 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:56 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:56.895+0000 7fe51faec040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:19:57.027 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:56 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:56.958+0000 7fe51faec040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:19:57.050 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:56.775+0000 7f918969e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:19:57.050 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:56.836+0000 7f918969e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:19:57.050 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:56.982+0000 7f918969e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T06:19:57.050 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.049+0000 7f918969e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:19:57.185 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:19:57.185 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":69,"fsid":"bd41205c-73f2-11ef-bceb-c7b262605968","created":"2024-09-16T06:14:07.938544+0000","modified":"2024-09-16T06:19:34.246886+0000","last_up_change":"2024-09-16T06:19:08.945843+0000","last_in_change":"2024-09-16T06:18:51.658721+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:17:08.572382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-16T06:19:19.777750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-16T06:19:20.011798+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T06:19:21.676347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T06:19:23.647060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T06:19:25.705633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0a4292e-21dc-43b1-9cfa-0d5de57a2484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6803","nonce":62621509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6805","nonce":62621509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6809","nonce":62621509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6807","nonce":62621509}]},"public_addr":"172.21.15.190:6803/62621509","cluster_addr":"172.21.15.190:6805/62621509","heartbeat_back_addr":"172.21.15.190:6809/62621509","heartbeat_front_addr":"172.21.15.190:6807/62621509","state":["exists","up"]},{"osd":1,"uuid":"52400e70-bb15-46cf-9280-8b9c44a9c16b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6811","nonce":525777266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6813","nonce":525777266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6817","nonce":525777266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6815","nonce":525777266}]},"public_addr":"172.21.15.190:6811/525777266","cluster_addr":"172.21.15.190:6813/525777266","heartbeat_back_addr":"172.21.15.190:6817/525777266","heartbeat_front_addr":"172.21.15.190:6815/525777266","state":["exists","up"]},{"osd":2,"uuid":"46b8a2a4-b143-4784-8780-0ed28f8e0276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6819","nonce":3137551404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6821","nonce":3137551404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6825","nonce":3137551404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6823","nonce":3137551404}]},"public_addr":"172.21.15.190:6819/3137551404","cluster_addr":"172.21.15.190:6821/3137551404","heartbeat_back_addr":"172.21.15.190:6825/3137551404","heartbeat_front_addr":"172.21.15.190:6823/3137551404","state":["exists","up"]},{"osd":3,"uuid":"3cbdf234-9282-4ab1-a636-03abcecece3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6827","nonce":1507636168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6829","nonce":1507636168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6833","nonce":1507636168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6831","nonce":1507636168}]},"public_addr":"172.21.15.190:6827/1507636168","cluster_addr":"172.21.15.190:6829/1507636168","heartbeat_back_addr":"172.21.15.190:6833/1507636168","heartbeat_front_addr":"172.21.15.190:6831/1507636168","state":["exists","up"]},{"osd":4,"uuid":"fd5c0769-e97a-483f-94a3-73b9df643a92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6801","nonce":2723520790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6803","nonce":2723520790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6807","nonce":2723520790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6805","nonce":2723520790}]},"public_addr":"172.21.15.196:6801/2723520790","cluster_addr":"172.21.15.196:6803/2723520790","heartbeat_back_addr":"172.21.15.196:6807/2723520790","heartbeat_front_addr":"172.21.15.196:6805/2723520790","state":["exists","up"]},{"osd":5,"uuid":"c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6809","nonce":561616070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6811","nonce":561616070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6815","nonce":561616070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6813","nonce":561616070}]},"public_addr":"172.21.15.196:6809/561616070","cluster_addr":"172.21.15.196:6811/561616070","heartbeat_back_addr":"172.21.15.196:6815/561616070","heartbeat_front_addr":"172.21.15.196:6813/561616070","state":["exists","up"]},{"osd":6,"uuid":"ac660cff-3f86-4a98-8cbc-97478337878b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6817","nonce":3795627894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6819","nonce":3795627894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6823","nonce":3795627894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6821","nonce":3795627894}]},"public_addr":"172.21.15.196:6817/3795627894","cluster_addr":"172.21.15.196:6819/3795627894","heartbeat_back_addr":"172.21.15.196:6823/3795627894","heartbeat_front_addr":"172.21.15.196:6821/3795627894","state":["exists","up"]},{"osd":7,"uuid":"07420cdf-64f6-4c03-878e-9b31d6b466e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6825","nonce":2986179076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6827","nonce":2986179076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6831","nonce":2986179076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6829","nonce":2986179076}]},"public_addr":"172.21.15.196:6825/2986179076","cluster_addr":"172.21.15.196:6827/2986179076","heartbeat_back_addr":"172.21.15.196:6831/2986179076","heartbeat_front_addr":"172.21.15.196:6829/2986179076","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:16:16.262056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:16:40.843599+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:05.820305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:30.199586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:52.476721+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:18:17.390373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:18:42.103890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:19:07.208019+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:0/522299671":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6801/2541445906":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/3600548341":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/3341383511":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/2985493186":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/3599123797":"2024-09-17T06:14:31.152761+0000","172.21.15.190:6800/2541445906":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6801/1748866605":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/4051857884":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6800/1748866605":"2024-09-17T06:14:31.152761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:19:57.329 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2163092107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:19:57.329 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.108+0000 7f918969e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:19:57.329 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.263+0000 7f918969e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:19:57.346 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2163092107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:19:57.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2163092107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:19:57.347 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.110+0000 7fe51faec040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T06:19:57.347 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.178+0000 7fe51faec040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:19:57.347 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.245+0000 7fe51faec040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:19:57.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.328+0000 7f918969e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:19:57.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.394+0000 7f918969e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:19:57.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.478+0000 7f918969e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:19:57.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.536+0000 7f918969e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:19:57.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:57.594+0000 7f918969e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:19:57.620 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.403+0000 7fe51faec040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:19:57.620 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.471+0000 7fe51faec040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:19:57.620 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.537+0000 7fe51faec040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:19:57.620 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.619+0000 7fe51faec040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:19:57.752 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T06:19:57.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T06:19:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:19:58.082 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.680+0000 7fe51faec040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:19:58.082 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:57.739+0000 7fe51faec040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:19:58.760 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[25888]: Standby manager daemon x restarted 2024-09-16T06:19:58.760 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[25888]: Standby manager daemon x started 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[25888]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[39731]: Standby manager daemon x restarted 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[39731]: Standby manager daemon x started 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T06:19:58.761 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-mon[39731]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:58.562+0000 7fe51faec040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:19:58] ENGINE Bus STARTING 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: CherryPy Checker: 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: The Application mounted at '' has an empty config. 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:19:58] ENGINE Serving on http://:::9283 2024-09-16T06:19:58.762 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:58 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:19:58] ENGINE Bus STARTED 2024-09-16T06:19:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-mon[30578]: Standby manager daemon x restarted 2024-09-16T06:19:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-mon[30578]: Standby manager daemon x started 2024-09-16T06:19:58.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-mon[30578]: from='mgr.? 172.21.15.196:0/3162914721' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:58.364+0000 7f918969e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: [16/Sep/2024:06:19:58] ENGINE Bus STARTING 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: CherryPy Checker: 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: The Application mounted at '' has an empty config. 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: [16/Sep/2024:06:19:58] ENGINE Serving on http://:::9283 2024-09-16T06:19:58.766 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:19:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x[32657]: [16/Sep/2024:06:19:58] ENGINE Bus STARTED 2024-09-16T06:19:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T06:19:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: Active manager daemon y restarted 2024-09-16T06:19:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: Activating manager daemon y 2024-09-16T06:19:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T06:19:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: mgrmap e19: y(active, starting, since 0.0100923s), standbys: x 2024-09-16T06:19:59.597 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:59.598 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: Manager daemon y is now available 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: Active manager daemon y restarted 2024-09-16T06:19:59.599 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: Activating manager daemon y 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: mgrmap e19: y(active, starting, since 0.0100923s), standbys: x 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:19:59.600 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: Manager daemon y is now available 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T06:19:59.601 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T06:19:59.737 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T06:19:59.737 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: Active manager daemon y restarted 2024-09-16T06:19:59.737 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: Activating manager daemon y 2024-09-16T06:19:59.737 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: mgrmap e19: y(active, starting, since 0.0100923s), standbys: x 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T06:19:59.738 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: Manager daemon y is now available 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T06:19:59.739 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:19:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T06:20:00.004 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:20:00.004 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":70,"fsid":"bd41205c-73f2-11ef-bceb-c7b262605968","created":"2024-09-16T06:14:07.938544+0000","modified":"2024-09-16T06:19:58.565490+0000","last_up_change":"2024-09-16T06:19:08.945843+0000","last_in_change":"2024-09-16T06:18:51.658721+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:17:08.572382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-16T06:19:19.777750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-16T06:19:20.011798+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T06:19:21.676347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T06:19:23.647060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T06:19:25.705633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0a4292e-21dc-43b1-9cfa-0d5de57a2484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6803","nonce":62621509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6805","nonce":62621509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6809","nonce":62621509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":62621509},{"type":"v1","addr":"172.21.15.190:6807","nonce":62621509}]},"public_addr":"172.21.15.190:6803/62621509","cluster_addr":"172.21.15.190:6805/62621509","heartbeat_back_addr":"172.21.15.190:6809/62621509","heartbeat_front_addr":"172.21.15.190:6807/62621509","state":["exists","up"]},{"osd":1,"uuid":"52400e70-bb15-46cf-9280-8b9c44a9c16b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6811","nonce":525777266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6813","nonce":525777266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6817","nonce":525777266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":525777266},{"type":"v1","addr":"172.21.15.190:6815","nonce":525777266}]},"public_addr":"172.21.15.190:6811/525777266","cluster_addr":"172.21.15.190:6813/525777266","heartbeat_back_addr":"172.21.15.190:6817/525777266","heartbeat_front_addr":"172.21.15.190:6815/525777266","state":["exists","up"]},{"osd":2,"uuid":"46b8a2a4-b143-4784-8780-0ed28f8e0276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6819","nonce":3137551404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6821","nonce":3137551404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6825","nonce":3137551404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3137551404},{"type":"v1","addr":"172.21.15.190:6823","nonce":3137551404}]},"public_addr":"172.21.15.190:6819/3137551404","cluster_addr":"172.21.15.190:6821/3137551404","heartbeat_back_addr":"172.21.15.190:6825/3137551404","heartbeat_front_addr":"172.21.15.190:6823/3137551404","state":["exists","up"]},{"osd":3,"uuid":"3cbdf234-9282-4ab1-a636-03abcecece3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6827","nonce":1507636168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6829","nonce":1507636168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6833","nonce":1507636168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1507636168},{"type":"v1","addr":"172.21.15.190:6831","nonce":1507636168}]},"public_addr":"172.21.15.190:6827/1507636168","cluster_addr":"172.21.15.190:6829/1507636168","heartbeat_back_addr":"172.21.15.190:6833/1507636168","heartbeat_front_addr":"172.21.15.190:6831/1507636168","state":["exists","up"]},{"osd":4,"uuid":"fd5c0769-e97a-483f-94a3-73b9df643a92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6801","nonce":2723520790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6803","nonce":2723520790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6807","nonce":2723520790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2723520790},{"type":"v1","addr":"172.21.15.196:6805","nonce":2723520790}]},"public_addr":"172.21.15.196:6801/2723520790","cluster_addr":"172.21.15.196:6803/2723520790","heartbeat_back_addr":"172.21.15.196:6807/2723520790","heartbeat_front_addr":"172.21.15.196:6805/2723520790","state":["exists","up"]},{"osd":5,"uuid":"c8f0a8a2-2c4d-4db8-bee5-db581e6fdbbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6809","nonce":561616070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6811","nonce":561616070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6815","nonce":561616070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":561616070},{"type":"v1","addr":"172.21.15.196:6813","nonce":561616070}]},"public_addr":"172.21.15.196:6809/561616070","cluster_addr":"172.21.15.196:6811/561616070","heartbeat_back_addr":"172.21.15.196:6815/561616070","heartbeat_front_addr":"172.21.15.196:6813/561616070","state":["exists","up"]},{"osd":6,"uuid":"ac660cff-3f86-4a98-8cbc-97478337878b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6817","nonce":3795627894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6819","nonce":3795627894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6823","nonce":3795627894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3795627894},{"type":"v1","addr":"172.21.15.196:6821","nonce":3795627894}]},"public_addr":"172.21.15.196:6817/3795627894","cluster_addr":"172.21.15.196:6819/3795627894","heartbeat_back_addr":"172.21.15.196:6823/3795627894","heartbeat_front_addr":"172.21.15.196:6821/3795627894","state":["exists","up"]},{"osd":7,"uuid":"07420cdf-64f6-4c03-878e-9b31d6b466e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6825","nonce":2986179076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6827","nonce":2986179076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6831","nonce":2986179076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2986179076},{"type":"v1","addr":"172.21.15.196:6829","nonce":2986179076}]},"public_addr":"172.21.15.196:6825/2986179076","cluster_addr":"172.21.15.196:6827/2986179076","heartbeat_back_addr":"172.21.15.196:6831/2986179076","heartbeat_front_addr":"172.21.15.196:6829/2986179076","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:16:16.262056+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:16:40.843599+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:05.820305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:30.199586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:17:52.476721+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:18:17.390373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:18:42.103890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:19:07.208019+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:0/1048924925":"2024-09-17T06:19:58.565450+0000","172.21.15.190:0/2924209252":"2024-09-17T06:19:58.565450+0000","172.21.15.190:6800/1748866605":"2024-09-17T06:14:31.152761+0000","172.21.15.190:6801/1748866605":"2024-09-17T06:14:31.152761+0000","172.21.15.190:6801/2827383855":"2024-09-17T06:19:58.565450+0000","172.21.15.190:0/3599123797":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/2985493186":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/3385091135":"2024-09-17T06:19:58.565450+0000","172.21.15.190:0/4051857884":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6800/2827383855":"2024-09-17T06:19:58.565450+0000","172.21.15.190:0/3341383511":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/3600548341":"2024-09-17T06:14:31.152761+0000","172.21.15.190:0/2689243789":"2024-09-17T06:19:58.565450+0000","172.21.15.190:6801/2541445906":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/522299671":"2024-09-17T06:14:54.263861+0000","172.21.15.190:6800/2541445906":"2024-09-17T06:14:54.263861+0000","172.21.15.190:0/3271532069":"2024-09-17T06:19:58.565450+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:19:59.980+0000 7fe47d7e6640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:06:19:59] ENGINE Error in HTTPServer.serve 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: Traceback (most recent call last): 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._connections.run(self.expiration_interval) 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._run(expiration_interval) 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:20:00.322 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: s = self.context.wrap_socket( 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: return self.sslsocket_class._create( 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self.do_handshake() 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: self._sslobj.do_handshake() 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:20:00.323 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:19:59 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[39731]: mgrmap e20: y(active, since 1.01245s), standbys: x 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[39731]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[39731]: overall HEALTH_OK 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/540676729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[25888]: mgrmap e20: y(active, since 1.01245s), standbys: x 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[25888]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[25888]: overall HEALTH_OK 2024-09-16T06:20:00.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:00 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/540676729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:20:01.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:00 smithi196 ceph-mon[30578]: mgrmap e20: y(active, since 1.01245s), standbys: x 2024-09-16T06:20:01.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:00 smithi196 ceph-mon[30578]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:01.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:00 smithi196 ceph-mon[30578]: overall HEALTH_OK 2024-09-16T06:20:01.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:00 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/540676729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T06:20:01.150 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T06:20:01.150 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T06:20:01.151 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T06:20:01.151 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T06:20:01.151 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T06:20:01.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T06:20:01.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T06:20:01.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T06:20:01.577 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.579 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.581 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: [16/Sep/2024:06:19:59] ENGINE Bus STARTING 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: [16/Sep/2024:06:19:59] ENGINE Serving on https://172.21.15.190:7150 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: [16/Sep/2024:06:19:59] ENGINE Error in HTTPServer.serve 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout: Traceback (most recent call last): 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._connections.run(self.expiration_interval) 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._run(expiration_interval) 2024-09-16T06:20:01.586 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: s = self.context.wrap_socket( 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: return self.sslsocket_class._create( 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: self.do_handshake() 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: self._sslobj.do_handshake() 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:20:01.587 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: [16/Sep/2024:06:20:00] ENGINE Serving on http://172.21.15.190:8765 2024-09-16T06:20:01.588 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: [16/Sep/2024:06:20:00] ENGINE Bus STARTED 2024-09-16T06:20:01.588 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:01.588 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[25888]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T06:20:01.588 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.594 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.602 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.603 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.609 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: [16/Sep/2024:06:19:59] ENGINE Bus STARTING 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: [16/Sep/2024:06:19:59] ENGINE Serving on https://172.21.15.190:7150 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: [16/Sep/2024:06:19:59] ENGINE Error in HTTPServer.serve 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout: Traceback (most recent call last): 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout: self._connections.run(self.expiration_interval) 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:20:01.847 INFO:journalctl@ceph.mon.c.smithi190.stdout: self._run(expiration_interval) 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: s = self.context.wrap_socket( 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: return self.sslsocket_class._create( 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: self.do_handshake() 2024-09-16T06:20:01.848 INFO:journalctl@ceph.mon.c.smithi190.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:20:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout: self._sslobj.do_handshake() 2024-09-16T06:20:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:20:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: [16/Sep/2024:06:20:00] ENGINE Serving on http://172.21.15.190:8765 2024-09-16T06:20:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: [16/Sep/2024:06:20:00] ENGINE Bus STARTED 2024-09-16T06:20:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:01.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:01 smithi190 ceph-mon[39731]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T06:20:02.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: [16/Sep/2024:06:19:59] ENGINE Bus STARTING 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: [16/Sep/2024:06:19:59] ENGINE Serving on https://172.21.15.190:7150 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: [16/Sep/2024:06:19:59] ENGINE Error in HTTPServer.serve 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: Traceback (most recent call last): 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._connections.run(self.expiration_interval) 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._run(expiration_interval) 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T06:20:02.016 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: s = self.context.wrap_socket( 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: return self.sslsocket_class._create( 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: self.do_handshake() 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._sslobj.do_handshake() 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: [16/Sep/2024:06:20:00] ENGINE Serving on http://172.21.15.190:8765 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: [16/Sep/2024:06:20:00] ENGINE Bus STARTED 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:02.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:01 smithi196 ceph-mon[30578]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T06:20:02.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:02.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:02 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:02 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:20:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi190 to 3760M 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:20:03.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi190 to 3760M 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi190 to 3760M 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:20:03.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.943 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:04.944 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:04 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:05.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:05.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: Updating smithi190:/etc/ceph/ceph.conf 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:05.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:04 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:06.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:20:06.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:20:06.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:20:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:20:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:20:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:20:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: Deploying daemon alertmanager.a on smithi190 2024-09-16T06:20:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:05 smithi196 ceph-mon[30578]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: Deploying daemon alertmanager.a on smithi190 2024-09-16T06:20:06.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[39731]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Updating smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Updating smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: Deploying daemon alertmanager.a on smithi190 2024-09-16T06:20:06.027 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:05 smithi190 ceph-mon[25888]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-16T06:20:07.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:07 smithi190 ceph-mon[25888]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T06:20:07.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:07 smithi190 ceph-mon[39731]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T06:20:08.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:07 smithi196 ceph-mon[30578]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T06:20:08.346 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:07 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:20:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:20:10.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:09 smithi196 ceph-mon[30578]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T06:20:10.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:09 smithi190 ceph-mon[39731]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T06:20:10.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:09 smithi190 ceph-mon[25888]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T06:20:11.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:11 smithi190 ceph-mon[25888]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T06:20:11.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:11 smithi190 ceph-mon[39731]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T06:20:12.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:11 smithi196 ceph-mon[30578]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T06:20:13.692 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:20:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:20:14.013 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:13 smithi190 ceph-mon[25888]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T06:20:14.013 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:13 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:14.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:13 smithi190 ceph-mon[39731]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T06:20:14.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:13 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:14.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:13 smithi196 ceph-mon[30578]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T06:20:14.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:13 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:14.034 INFO:teuthology.orchestra.run.smithi190.stdout:167503724567 2024-09-16T06:20:14.034 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T06:20:14.908 INFO:teuthology.orchestra.run.smithi190.stdout:201863462931 2024-09-16T06:20:14.909 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T06:20:15.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:14 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:15.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:14 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:15.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:14 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:15.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:15 smithi190 ceph-mon[25888]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T06:20:15.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:15 smithi190 ceph-mon[39731]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T06:20:16.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:15 smithi196 ceph-mon[30578]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T06:20:16.989 INFO:teuthology.orchestra.run.smithi190.stdout:111669149731 2024-09-16T06:20:16.990 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T06:20:18.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:17 smithi196 ceph-mon[30578]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:18.033 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:17 smithi190 ceph-mon[39731]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:18.033 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:17 smithi190 ceph-mon[25888]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:18.033 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:20:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:20:19.621 INFO:teuthology.orchestra.run.smithi190.stdout:55834574893 2024-09-16T06:20:19.621 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T06:20:19.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:19 smithi190 ceph-mon[25888]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:19.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:19 smithi190 ceph-mon[39731]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:20.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:19 smithi196 ceph-mon[30578]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:20.382 INFO:teuthology.orchestra.run.smithi190.stdout:77309411369 2024-09-16T06:20:20.382 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T06:20:20.623 INFO:teuthology.orchestra.run.smithi190.stdout:133143986205 2024-09-16T06:20:20.623 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T06:20:21.959 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:21 smithi190 ceph-mon[25888]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:21.959 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:21 smithi190 ceph-mon[39731]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:22.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:21 smithi196 ceph-mon[30578]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:22.910 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:22 smithi190 ceph-mon[25888]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:22.911 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:22 smithi190 ceph-mon[39731]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:23.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:22 smithi196 ceph-mon[30578]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:23.228 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:23.228 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:23.260 INFO:teuthology.orchestra.run.smithi190.stdout:34359738418 2024-09-16T06:20:23.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T06:20:23.630 INFO:teuthology.orchestra.run.smithi190.stdout:236223201295 2024-09-16T06:20:23.630 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T06:20:23.641 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:23.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:20:23 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:20:24.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:23 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:24.073 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:23 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:24.074 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:23 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:24.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:24 smithi190 ceph-mon[39731]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:24.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:24 smithi190 ceph-mon[25888]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:25.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:24 smithi196 ceph-mon[30578]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:27.490 INFO:teuthology.orchestra.run.smithi190.stdout:167503724571 2024-09-16T06:20:27.729 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:27.729 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:27.729 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:27.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:27 smithi190 ceph-mon[39731]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:27.731 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:27 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2340652552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T06:20:27.731 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:27 smithi190 ceph-mon[25888]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:27.731 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:27 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2340652552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T06:20:27.903 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:28.005 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:27 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:20:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:20:28.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:27 smithi196 ceph-mon[30578]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:28.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:27 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2340652552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T06:20:29.128 INFO:teuthology.orchestra.run.smithi190.stdout:201863462934 2024-09-16T06:20:29.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724567 got 167503724571 for osd.5 2024-09-16T06:20:29.371 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:29.927 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:29 smithi190 ceph-mon[39731]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:29.927 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:29 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:29.927 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:29 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/1504955219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T06:20:29.927 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:29 smithi190 ceph-mon[25888]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:29.927 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:29 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:29.927 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:29 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1504955219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T06:20:30.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:29 smithi196 ceph-mon[30578]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:30.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:29 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:30.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:29 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/1504955219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T06:20:30.690 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:31.074 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462931 got 201863462934 for osd.6 2024-09-16T06:20:31.074 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:31.925 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:31 smithi190 ceph-mon[25888]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:31.925 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:31 smithi190 ceph-mon[39731]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:32.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:31 smithi196 ceph-mon[30578]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:32.500 INFO:teuthology.orchestra.run.smithi190.stdout:55834574896 2024-09-16T06:20:32.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:32 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2305428446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T06:20:32.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:32 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2305428446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T06:20:33.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:32 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2305428446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T06:20:33.309 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574896 for osd.1 2024-09-16T06:20:33.309 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:33.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:33 smithi196 ceph-mon[30578]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:33.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:20:33 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:20:34.039 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:33 smithi190 ceph-mon[39731]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:34.039 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:33 smithi190 ceph-mon[25888]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:34.929 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:34 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:34.929 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:34 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:35.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:34 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:35.182 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 podman[84159]: 2024-09-16 06:20:35.115751794 +0000 UTC m=+7.196215990 volume create f3b089a94b884ca9c7a4a35a574c66675817ea921ec67b1c8bc48b506d6ec4f8 2024-09-16T06:20:35.182 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 podman[84159]: 2024-09-16 06:20:35.182589223 +0000 UTC m=+7.263053374 container create cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:20:35.601 INFO:teuthology.orchestra.run.smithi190.stdout:111669149736 2024-09-16T06:20:35.819 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-mon[25888]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:35.819 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/447573803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T06:20:35.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-mon[39731]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:35.820 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/447573803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T06:20:35.820 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 podman[84159]: 2024-09-16 06:20:35.711898735 +0000 UTC m=+7.792362888 container init cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:20:35.820 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 podman[84159]: 2024-09-16 06:20:35.717127534 +0000 UTC m=+7.797591687 container start cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:20:35.820 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.729Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T06:20:35.820 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.729Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T06:20:35.820 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.730Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.190 port=9094 2024-09-16T06:20:35.821 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.731Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T06:20:35.821 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.754Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T06:20:35.821 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.754Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T06:20:35.821 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.757Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T06:20:35.821 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:35.757Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T06:20:35.821 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 bash[84159]: cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f 2024-09-16T06:20:36.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:35 smithi196 ceph-mon[30578]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:36.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:35 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/447573803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T06:20:36.096 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:35 smithi190 systemd[1]: Started Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:20:36.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:35 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:20:35] ENGINE Bus STOPPING 2024-09-16T06:20:36.496 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149731 got 111669149736 for osd.3 2024-09-16T06:20:36.496 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:36.707 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:20:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T06:20:36.708 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:20:36] ENGINE Bus STOPPED 2024-09-16T06:20:36.708 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:20:36] ENGINE Bus STARTING 2024-09-16T06:20:36.708 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:20:36] ENGINE Serving on http://:::9283 2024-09-16T06:20:36.708 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:20:36] ENGINE Bus STARTED 2024-09-16T06:20:37.020 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.020 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.021 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: Deploying daemon grafana.a on smithi196 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[25888]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T06:20:37.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.023 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: Deploying daemon grafana.a on smithi196 2024-09-16T06:20:37.023 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:36 smithi190 ceph-mon[39731]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:37.103 INFO:teuthology.orchestra.run.smithi190.stdout:34359738422 2024-09-16T06:20:37.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: Deploying daemon grafana.a on smithi196 2024-09-16T06:20:37.266 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:36 smithi196 ceph-mon[30578]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:37.915 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:37 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:37.732Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000528377s 2024-09-16T06:20:37.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738422 for osd.0 2024-09-16T06:20:37.967 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:38.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:37 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2939004227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T06:20:38.347 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:37 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2939004227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T06:20:38.347 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:37 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2939004227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T06:20:38.348 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:37 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:20:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:20:39.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:39 smithi190 ceph-mon[39731]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:39.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:39 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:39.845 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:39 smithi190 ceph-mon[25888]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:39.845 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:39 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:40.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:39 smithi196 ceph-mon[30578]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:40.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:39 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:40.462 INFO:teuthology.orchestra.run.smithi190.stdout:77309411373 2024-09-16T06:20:40.744 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:40 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/3333039367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T06:20:40.745 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:40 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/3333039367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T06:20:40.844 INFO:teuthology.orchestra.run.smithi190.stdout:236223201300 2024-09-16T06:20:41.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:40 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/3333039367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T06:20:41.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201295 got 236223201300 for osd.7 2024-09-16T06:20:41.416 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:41.507 INFO:teuthology.orchestra.run.smithi190.stdout:133143986211 2024-09-16T06:20:41.778 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:41 smithi190 ceph-mon[25888]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:41.778 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:41 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1302503632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T06:20:41.778 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:41 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/1673254473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T06:20:41.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:41 smithi190 ceph-mon[39731]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:41 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/1302503632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T06:20:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:41 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/1673254473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T06:20:42.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:41 smithi196 ceph-mon[30578]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:42.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:41 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/1302503632' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T06:20:42.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:41 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/1673254473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T06:20:42.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986205 got 133143986211 for osd.4 2024-09-16T06:20:42.112 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:42.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411373 for osd.2 2024-09-16T06:20:42.591 DEBUG:teuthology.parallel:result is None 2024-09-16T06:20:42.591 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T06:20:42.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T06:20:42.796 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:43.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:20:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:20:44.012 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:43 smithi190 ceph-mon[25888]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:44.012 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:43 smithi190 ceph-mon[39731]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:44.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:43 smithi196 ceph-mon[30578]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:44.700 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:20:44.702 INFO:teuthology.orchestra.run.smithi190.stderr:dumped all 2024-09-16T06:20:44.748 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:44 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:44.749 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:44 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:45.020 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:44 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:45.020 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:44 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:45.020 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:44 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:45.020 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:44 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:45.419 INFO:teuthology.orchestra.run.smithi190.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-16T06:20:44.594958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1029,"num_read_kb":872,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":232164,"kb_used_data":7820,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749696284,"statfs":{"total":767926730752,"available":767688994816,"internally_reserved":0,"allocated":8007680,"data_stored":4333272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542507+0000","last_change":"2024-09-16T06:19:27.781523+0000","last_active":"2024-09-16T06:19:34.542507+0000","last_peered":"2024-09-16T06:19:34.542507+0000","last_clean":"2024-09-16T06:19:34.542507+0000","last_became_active":"2024-09-16T06:19:27.780398+0000","last_became_peered":"2024-09-16T06:19:27.780398+0000","last_unstale":"2024-09-16T06:19:34.542507+0000","last_undegraded":"2024-09-16T06:19:34.542507+0000","last_fullsized":"2024-09-16T06:19:34.542507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:34.851192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.544026+0000","last_change":"2024-09-16T06:19:21.393343+0000","last_active":"2024-09-16T06:19:34.544026+0000","last_peered":"2024-09-16T06:19:34.544026+0000","last_clean":"2024-09-16T06:19:34.544026+0000","last_became_active":"2024-09-16T06:19:21.393038+0000","last_became_peered":"2024-09-16T06:19:21.393038+0000","last_unstale":"2024-09-16T06:19:34.544026+0000","last_undegraded":"2024-09-16T06:19:34.544026+0000","last_fullsized":"2024-09-16T06:19:34.544026+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:50.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542899+0000","last_change":"2024-09-16T06:19:23.408766+0000","last_active":"2024-09-16T06:19:34.542899+0000","last_peered":"2024-09-16T06:19:34.542899+0000","last_clean":"2024-09-16T06:19:34.542899+0000","last_became_active":"2024-09-16T06:19:23.408446+0000","last_became_peered":"2024-09-16T06:19:23.408446+0000","last_unstale":"2024-09-16T06:19:34.542899+0000","last_undegraded":"2024-09-16T06:19:34.542899+0000","last_fullsized":"2024-09-16T06:19:34.542899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:23.425549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541740+0000","last_change":"2024-09-16T06:19:25.409178+0000","last_active":"2024-09-16T06:19:34.541740+0000","last_peered":"2024-09-16T06:19:34.541740+0000","last_clean":"2024-09-16T06:19:34.541740+0000","last_became_active":"2024-09-16T06:19:25.408903+0000","last_became_peered":"2024-09-16T06:19:25.408903+0000","last_unstale":"2024-09-16T06:19:34.541740+0000","last_undegraded":"2024-09-16T06:19:34.541740+0000","last_fullsized":"2024-09-16T06:19:34.541740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:30.736315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543873+0000","last_change":"2024-09-16T06:19:21.405390+0000","last_active":"2024-09-16T06:19:34.543873+0000","last_peered":"2024-09-16T06:19:34.543873+0000","last_clean":"2024-09-16T06:19:34.543873+0000","last_became_active":"2024-09-16T06:19:21.405007+0000","last_became_peered":"2024-09-16T06:19:21.405007+0000","last_unstale":"2024-09-16T06:19:34.543873+0000","last_undegraded":"2024-09-16T06:19:34.543873+0000","last_fullsized":"2024-09-16T06:19:34.543873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:56.931842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541752+0000","last_change":"2024-09-16T06:19:23.406457+0000","last_active":"2024-09-16T06:19:34.541752+0000","last_peered":"2024-09-16T06:19:34.541752+0000","last_clean":"2024-09-16T06:19:34.541752+0000","last_became_active":"2024-09-16T06:19:23.406129+0000","last_became_peered":"2024-09-16T06:19:23.406129+0000","last_unstale":"2024-09-16T06:19:34.541752+0000","last_undegraded":"2024-09-16T06:19:34.541752+0000","last_fullsized":"2024-09-16T06:19:34.541752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:39.561282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253043+0000","last_change":"2024-09-16T06:19:25.409113+0000","last_active":"2024-09-16T06:19:34.253043+0000","last_peered":"2024-09-16T06:19:34.253043+0000","last_clean":"2024-09-16T06:19:34.253043+0000","last_became_active":"2024-09-16T06:19:25.408801+0000","last_became_peered":"2024-09-16T06:19:25.408801+0000","last_unstale":"2024-09-16T06:19:34.253043+0000","last_undegraded":"2024-09-16T06:19:34.253043+0000","last_fullsized":"2024-09-16T06:19:34.253043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:09.045956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542235+0000","last_change":"2024-09-16T06:19:27.417778+0000","last_active":"2024-09-16T06:19:34.542235+0000","last_peered":"2024-09-16T06:19:34.542235+0000","last_clean":"2024-09-16T06:19:34.542235+0000","last_became_active":"2024-09-16T06:19:27.417541+0000","last_became_peered":"2024-09-16T06:19:27.417541+0000","last_unstale":"2024-09-16T06:19:34.542235+0000","last_undegraded":"2024-09-16T06:19:34.542235+0000","last_fullsized":"2024-09-16T06:19:34.542235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:34.254452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542924+0000","last_change":"2024-09-16T06:19:21.394356+0000","last_active":"2024-09-16T06:19:34.542924+0000","last_peered":"2024-09-16T06:19:34.542924+0000","last_clean":"2024-09-16T06:19:34.542924+0000","last_became_active":"2024-09-16T06:19:21.394112+0000","last_became_peered":"2024-09-16T06:19:21.394112+0000","last_unstale":"2024-09-16T06:19:34.542924+0000","last_undegraded":"2024-09-16T06:19:34.542924+0000","last_fullsized":"2024-09-16T06:19:34.542924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:12.607205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542163+0000","last_change":"2024-09-16T06:19:23.410576+0000","last_active":"2024-09-16T06:19:34.542163+0000","last_peered":"2024-09-16T06:19:34.542163+0000","last_clean":"2024-09-16T06:19:34.542163+0000","last_became_active":"2024-09-16T06:19:23.410299+0000","last_became_peered":"2024-09-16T06:19:23.410299+0000","last_unstale":"2024-09-16T06:19:34.542163+0000","last_undegraded":"2024-09-16T06:19:34.542163+0000","last_fullsized":"2024-09-16T06:19:34.542163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:46.031823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542208+0000","last_change":"2024-09-16T06:19:25.409592+0000","last_active":"2024-09-16T06:19:34.542208+0000","last_peered":"2024-09-16T06:19:34.542208+0000","last_clean":"2024-09-16T06:19:34.542208+0000","last_became_active":"2024-09-16T06:19:25.409296+0000","last_became_peered":"2024-09-16T06:19:25.409296+0000","last_unstale":"2024-09-16T06:19:34.542208+0000","last_undegraded":"2024-09-16T06:19:34.542208+0000","last_fullsized":"2024-09-16T06:19:34.542208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:25.391296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251366+0000","last_change":"2024-09-16T06:19:27.421807+0000","last_active":"2024-09-16T06:19:34.251366+0000","last_peered":"2024-09-16T06:19:34.251366+0000","last_clean":"2024-09-16T06:19:34.251366+0000","last_became_active":"2024-09-16T06:19:27.421512+0000","last_became_peered":"2024-09-16T06:19:27.421512+0000","last_unstale":"2024-09-16T06:19:34.251366+0000","last_undegraded":"2024-09-16T06:19:34.251366+0000","last_fullsized":"2024-09-16T06:19:34.251366+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:53.714361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542666+0000","last_change":"2024-09-16T06:19:21.393902+0000","last_active":"2024-09-16T06:19:34.542666+0000","last_peered":"2024-09-16T06:19:34.542666+0000","last_clean":"2024-09-16T06:19:34.542666+0000","last_became_active":"2024-09-16T06:19:21.393626+0000","last_became_peered":"2024-09-16T06:19:21.393626+0000","last_unstale":"2024-09-16T06:19:34.542666+0000","last_undegraded":"2024-09-16T06:19:34.542666+0000","last_fullsized":"2024-09-16T06:19:34.542666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:09.812591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542473+0000","last_change":"2024-09-16T06:19:23.404483+0000","last_active":"2024-09-16T06:19:34.542473+0000","last_peered":"2024-09-16T06:19:34.542473+0000","last_clean":"2024-09-16T06:19:34.542473+0000","last_became_active":"2024-09-16T06:19:23.403509+0000","last_became_peered":"2024-09-16T06:19:23.403509+0000","last_unstale":"2024-09-16T06:19:34.542473+0000","last_undegraded":"2024-09-16T06:19:34.542473+0000","last_fullsized":"2024-09-16T06:19:34.542473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:41.966253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251488+0000","last_change":"2024-09-16T06:19:25.406805+0000","last_active":"2024-09-16T06:19:34.251488+0000","last_peered":"2024-09-16T06:19:34.251488+0000","last_clean":"2024-09-16T06:19:34.251488+0000","last_became_active":"2024-09-16T06:19:25.406523+0000","last_became_peered":"2024-09-16T06:19:25.406523+0000","last_unstale":"2024-09-16T06:19:34.251488+0000","last_undegraded":"2024-09-16T06:19:34.251488+0000","last_fullsized":"2024-09-16T06:19:34.251488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:41.291727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542358+0000","last_change":"2024-09-16T06:19:27.420331+0000","last_active":"2024-09-16T06:19:34.542358+0000","last_peered":"2024-09-16T06:19:34.542358+0000","last_clean":"2024-09-16T06:19:34.542358+0000","last_became_active":"2024-09-16T06:19:27.420052+0000","last_became_peered":"2024-09-16T06:19:27.420052+0000","last_unstale":"2024-09-16T06:19:34.542358+0000","last_undegraded":"2024-09-16T06:19:34.542358+0000","last_fullsized":"2024-09-16T06:19:34.542358+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:03.556292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":45,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542759+0000","last_change":"2024-09-16T06:19:23.409283+0000","last_active":"2024-09-16T06:19:34.542759+0000","last_peered":"2024-09-16T06:19:34.542759+0000","last_clean":"2024-09-16T06:19:34.542759+0000","last_became_active":"2024-09-16T06:19:23.408938+0000","last_became_peered":"2024-09-16T06:19:23.408938+0000","last_unstale":"2024-09-16T06:19:34.542759+0000","last_undegraded":"2024-09-16T06:19:34.542759+0000","last_fullsized":"2024-09-16T06:19:34.542759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:59:47.089761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542297+0000","last_change":"2024-09-16T06:19:21.399036+0000","last_active":"2024-09-16T06:19:34.542297+0000","last_peered":"2024-09-16T06:19:34.542297+0000","last_clean":"2024-09-16T06:19:34.542297+0000","last_became_active":"2024-09-16T06:19:21.398661+0000","last_became_peered":"2024-09-16T06:19:21.398661+0000","last_unstale":"2024-09-16T06:19:34.542297+0000","last_undegraded":"2024-09-16T06:19:34.542297+0000","last_fullsized":"2024-09-16T06:19:34.542297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:54.383899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253327+0000","last_change":"2024-09-16T06:19:25.409527+0000","last_active":"2024-09-16T06:19:34.253327+0000","last_peered":"2024-09-16T06:19:34.253327+0000","last_clean":"2024-09-16T06:19:34.253327+0000","last_became_active":"2024-09-16T06:19:25.409148+0000","last_became_peered":"2024-09-16T06:19:25.409148+0000","last_unstale":"2024-09-16T06:19:34.253327+0000","last_undegraded":"2024-09-16T06:19:34.253327+0000","last_fullsized":"2024-09-16T06:19:34.253327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:13:52.863638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254398+0000","last_change":"2024-09-16T06:19:27.426258+0000","last_active":"2024-09-16T06:19:34.254398+0000","last_peered":"2024-09-16T06:19:34.254398+0000","last_clean":"2024-09-16T06:19:34.254398+0000","last_became_active":"2024-09-16T06:19:27.425948+0000","last_became_peered":"2024-09-16T06:19:27.425948+0000","last_unstale":"2024-09-16T06:19:34.254398+0000","last_undegraded":"2024-09-16T06:19:34.254398+0000","last_fullsized":"2024-09-16T06:19:34.254398+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:36.470944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542362+0000","last_change":"2024-09-16T06:19:23.405156+0000","last_active":"2024-09-16T06:19:34.542362+0000","last_peered":"2024-09-16T06:19:34.542362+0000","last_clean":"2024-09-16T06:19:34.542362+0000","last_became_active":"2024-09-16T06:19:23.404618+0000","last_became_peered":"2024-09-16T06:19:23.404618+0000","last_unstale":"2024-09-16T06:19:34.542362+0000","last_undegraded":"2024-09-16T06:19:34.542362+0000","last_fullsized":"2024-09-16T06:19:34.542362+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:09.897750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254206+0000","last_change":"2024-09-16T06:19:21.399408+0000","last_active":"2024-09-16T06:19:34.254206+0000","last_peered":"2024-09-16T06:19:34.254206+0000","last_clean":"2024-09-16T06:19:34.254206+0000","last_became_active":"2024-09-16T06:19:21.399124+0000","last_became_peered":"2024-09-16T06:19:21.399124+0000","last_unstale":"2024-09-16T06:19:34.254206+0000","last_undegraded":"2024-09-16T06:19:34.254206+0000","last_fullsized":"2024-09-16T06:19:34.254206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:18.577960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551937+0000","last_change":"2024-09-16T06:19:25.406608+0000","last_active":"2024-09-16T06:20:39.551937+0000","last_peered":"2024-09-16T06:20:39.551937+0000","last_clean":"2024-09-16T06:20:39.551937+0000","last_became_active":"2024-09-16T06:19:25.406313+0000","last_became_peered":"2024-09-16T06:19:25.406313+0000","last_unstale":"2024-09-16T06:20:39.551937+0000","last_undegraded":"2024-09-16T06:20:39.551937+0000","last_fullsized":"2024-09-16T06:20:39.551937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:14.545971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542694+0000","last_change":"2024-09-16T06:19:27.413145+0000","last_active":"2024-09-16T06:19:34.542694+0000","last_peered":"2024-09-16T06:19:34.542694+0000","last_clean":"2024-09-16T06:19:34.542694+0000","last_became_active":"2024-09-16T06:19:27.412859+0000","last_became_peered":"2024-09-16T06:19:27.412859+0000","last_unstale":"2024-09-16T06:19:34.542694+0000","last_undegraded":"2024-09-16T06:19:34.542694+0000","last_fullsized":"2024-09-16T06:19:34.542694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:57.903427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254074+0000","last_change":"2024-09-16T06:19:23.406354+0000","last_active":"2024-09-16T06:19:34.254074+0000","last_peered":"2024-09-16T06:19:34.254074+0000","last_clean":"2024-09-16T06:19:34.254074+0000","last_became_active":"2024-09-16T06:19:23.405896+0000","last_became_peered":"2024-09-16T06:19:23.405896+0000","last_unstale":"2024-09-16T06:19:34.254074+0000","last_undegraded":"2024-09-16T06:19:34.254074+0000","last_fullsized":"2024-09-16T06:19:34.254074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:20.918608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253858+0000","last_change":"2024-09-16T06:19:21.400988+0000","last_active":"2024-09-16T06:19:34.253858+0000","last_peered":"2024-09-16T06:19:34.253858+0000","last_clean":"2024-09-16T06:19:34.253858+0000","last_became_active":"2024-09-16T06:19:21.400095+0000","last_became_peered":"2024-09-16T06:19:21.400095+0000","last_unstale":"2024-09-16T06:19:34.253858+0000","last_undegraded":"2024-09-16T06:19:34.253858+0000","last_fullsized":"2024-09-16T06:19:34.253858+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:59.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551898+0000","last_change":"2024-09-16T06:19:25.406216+0000","last_active":"2024-09-16T06:20:39.551898+0000","last_peered":"2024-09-16T06:20:39.551898+0000","last_clean":"2024-09-16T06:20:39.551898+0000","last_became_active":"2024-09-16T06:19:25.405934+0000","last_became_peered":"2024-09-16T06:19:25.405934+0000","last_unstale":"2024-09-16T06:20:39.551898+0000","last_undegraded":"2024-09-16T06:20:39.551898+0000","last_fullsized":"2024-09-16T06:20:39.551898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:30:43.587414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251715+0000","last_change":"2024-09-16T06:19:27.420455+0000","last_active":"2024-09-16T06:19:34.251715+0000","last_peered":"2024-09-16T06:19:34.251715+0000","last_clean":"2024-09-16T06:19:34.251715+0000","last_became_active":"2024-09-16T06:19:27.420216+0000","last_became_peered":"2024-09-16T06:19:27.420216+0000","last_unstale":"2024-09-16T06:19:34.251715+0000","last_undegraded":"2024-09-16T06:19:34.251715+0000","last_fullsized":"2024-09-16T06:19:34.251715+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:00.104650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542928+0000","last_change":"2024-09-16T06:19:23.410684+0000","last_active":"2024-09-16T06:19:34.542928+0000","last_peered":"2024-09-16T06:19:34.542928+0000","last_clean":"2024-09-16T06:19:34.542928+0000","last_became_active":"2024-09-16T06:19:23.410359+0000","last_became_peered":"2024-09-16T06:19:23.410359+0000","last_unstale":"2024-09-16T06:19:34.542928+0000","last_undegraded":"2024-09-16T06:19:34.542928+0000","last_fullsized":"2024-09-16T06:19:34.542928+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:59.730154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541931+0000","last_change":"2024-09-16T06:19:21.396555+0000","last_active":"2024-09-16T06:19:34.541931+0000","last_peered":"2024-09-16T06:19:34.541931+0000","last_clean":"2024-09-16T06:19:34.541931+0000","last_became_active":"2024-09-16T06:19:21.396224+0000","last_became_peered":"2024-09-16T06:19:21.396224+0000","last_unstale":"2024-09-16T06:19:34.541931+0000","last_undegraded":"2024-09-16T06:19:34.541931+0000","last_fullsized":"2024-09-16T06:19:34.541931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:28.462437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252002+0000","last_change":"2024-09-16T06:19:25.413057+0000","last_active":"2024-09-16T06:19:34.252002+0000","last_peered":"2024-09-16T06:19:34.252002+0000","last_clean":"2024-09-16T06:19:34.252002+0000","last_became_active":"2024-09-16T06:19:25.412711+0000","last_became_peered":"2024-09-16T06:19:25.412711+0000","last_unstale":"2024-09-16T06:19:34.252002+0000","last_undegraded":"2024-09-16T06:19:34.252002+0000","last_fullsized":"2024-09-16T06:19:34.252002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:10.860914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543126+0000","last_change":"2024-09-16T06:19:27.778907+0000","last_active":"2024-09-16T06:19:34.543126+0000","last_peered":"2024-09-16T06:19:34.543126+0000","last_clean":"2024-09-16T06:19:34.543126+0000","last_became_active":"2024-09-16T06:19:27.778591+0000","last_became_peered":"2024-09-16T06:19:27.778591+0000","last_unstale":"2024-09-16T06:19:34.543126+0000","last_undegraded":"2024-09-16T06:19:34.543126+0000","last_fullsized":"2024-09-16T06:19:34.543126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:29.207879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541926+0000","last_change":"2024-09-16T06:19:23.402100+0000","last_active":"2024-09-16T06:19:34.541926+0000","last_peered":"2024-09-16T06:19:34.541926+0000","last_clean":"2024-09-16T06:19:34.541926+0000","last_became_active":"2024-09-16T06:19:23.401840+0000","last_became_peered":"2024-09-16T06:19:23.401840+0000","last_unstale":"2024-09-16T06:19:34.541926+0000","last_undegraded":"2024-09-16T06:19:34.541926+0000","last_fullsized":"2024-09-16T06:19:34.541926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:04.039428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291207+0000","last_change":"2024-09-16T06:19:21.394696+0000","last_active":"2024-09-16T06:19:34.291207+0000","last_peered":"2024-09-16T06:19:34.291207+0000","last_clean":"2024-09-16T06:19:34.291207+0000","last_became_active":"2024-09-16T06:19:21.393723+0000","last_became_peered":"2024-09-16T06:19:21.393723+0000","last_unstale":"2024-09-16T06:19:34.291207+0000","last_undegraded":"2024-09-16T06:19:34.291207+0000","last_fullsized":"2024-09-16T06:19:34.291207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:24.190761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551960+0000","last_change":"2024-09-16T06:19:25.411198+0000","last_active":"2024-09-16T06:20:39.551960+0000","last_peered":"2024-09-16T06:20:39.551960+0000","last_clean":"2024-09-16T06:20:39.551960+0000","last_became_active":"2024-09-16T06:19:25.410886+0000","last_became_peered":"2024-09-16T06:19:25.410886+0000","last_unstale":"2024-09-16T06:20:39.551960+0000","last_undegraded":"2024-09-16T06:20:39.551960+0000","last_fullsized":"2024-09-16T06:20:39.551960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:53.964645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542112+0000","last_change":"2024-09-16T06:19:27.780730+0000","last_active":"2024-09-16T06:19:34.542112+0000","last_peered":"2024-09-16T06:19:34.542112+0000","last_clean":"2024-09-16T06:19:34.542112+0000","last_became_active":"2024-09-16T06:19:27.780369+0000","last_became_peered":"2024-09-16T06:19:27.780369+0000","last_unstale":"2024-09-16T06:19:34.542112+0000","last_undegraded":"2024-09-16T06:19:34.542112+0000","last_fullsized":"2024-09-16T06:19:34.542112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.083603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":49,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542428+0000","last_change":"2024-09-16T06:19:23.403324+0000","last_active":"2024-09-16T06:19:34.542428+0000","last_peered":"2024-09-16T06:19:34.542428+0000","last_clean":"2024-09-16T06:19:34.542428+0000","last_became_active":"2024-09-16T06:19:23.402993+0000","last_became_peered":"2024-09-16T06:19:23.402993+0000","last_unstale":"2024-09-16T06:19:34.542428+0000","last_undegraded":"2024-09-16T06:19:34.542428+0000","last_fullsized":"2024-09-16T06:19:34.542428+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:22:22.218412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253653+0000","last_change":"2024-09-16T06:19:21.395578+0000","last_active":"2024-09-16T06:19:34.253653+0000","last_peered":"2024-09-16T06:19:34.253653+0000","last_clean":"2024-09-16T06:19:34.253653+0000","last_became_active":"2024-09-16T06:19:21.395300+0000","last_became_peered":"2024-09-16T06:19:21.395300+0000","last_unstale":"2024-09-16T06:19:34.253653+0000","last_undegraded":"2024-09-16T06:19:34.253653+0000","last_fullsized":"2024-09-16T06:19:34.253653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:05.210405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541814+0000","last_change":"2024-09-16T06:19:25.407007+0000","last_active":"2024-09-16T06:19:34.541814+0000","last_peered":"2024-09-16T06:19:34.541814+0000","last_clean":"2024-09-16T06:19:34.541814+0000","last_became_active":"2024-09-16T06:19:25.406718+0000","last_became_peered":"2024-09-16T06:19:25.406718+0000","last_unstale":"2024-09-16T06:19:34.541814+0000","last_undegraded":"2024-09-16T06:19:34.541814+0000","last_fullsized":"2024-09-16T06:19:34.541814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:25.088858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542433+0000","last_change":"2024-09-16T06:19:27.423447+0000","last_active":"2024-09-16T06:19:34.542433+0000","last_peered":"2024-09-16T06:19:34.542433+0000","last_clean":"2024-09-16T06:19:34.542433+0000","last_became_active":"2024-09-16T06:19:27.423157+0000","last_became_peered":"2024-09-16T06:19:27.423157+0000","last_unstale":"2024-09-16T06:19:34.542433+0000","last_undegraded":"2024-09-16T06:19:34.542433+0000","last_fullsized":"2024-09-16T06:19:34.542433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:46.280210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":53,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543191+0000","last_change":"2024-09-16T06:19:23.405382+0000","last_active":"2024-09-16T06:19:34.543191+0000","last_peered":"2024-09-16T06:19:34.543191+0000","last_clean":"2024-09-16T06:19:34.543191+0000","last_became_active":"2024-09-16T06:19:23.404131+0000","last_became_peered":"2024-09-16T06:19:23.404131+0000","last_unstale":"2024-09-16T06:19:34.543191+0000","last_undegraded":"2024-09-16T06:19:34.543191+0000","last_fullsized":"2024-09-16T06:19:34.543191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:17.080529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"61'1","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542994+0000","last_change":"2024-09-16T06:19:21.395826+0000","last_active":"2024-09-16T06:19:34.542994+0000","last_peered":"2024-09-16T06:19:34.542994+0000","last_clean":"2024-09-16T06:19:34.542994+0000","last_became_active":"2024-09-16T06:19:21.395569+0000","last_became_peered":"2024-09-16T06:19:21.395569+0000","last_unstale":"2024-09-16T06:19:34.542994+0000","last_undegraded":"2024-09-16T06:19:34.542994+0000","last_fullsized":"2024-09-16T06:19:34.542994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:44.665552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290788+0000","last_change":"2024-09-16T06:19:25.408281+0000","last_active":"2024-09-16T06:19:34.290788+0000","last_peered":"2024-09-16T06:19:34.290788+0000","last_clean":"2024-09-16T06:19:34.290788+0000","last_became_active":"2024-09-16T06:19:25.407940+0000","last_became_peered":"2024-09-16T06:19:25.407940+0000","last_unstale":"2024-09-16T06:19:34.290788+0000","last_undegraded":"2024-09-16T06:19:34.290788+0000","last_fullsized":"2024-09-16T06:19:34.290788+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:55.862327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252994+0000","last_change":"2024-09-16T06:19:27.781363+0000","last_active":"2024-09-16T06:19:34.252994+0000","last_peered":"2024-09-16T06:19:34.252994+0000","last_clean":"2024-09-16T06:19:34.252994+0000","last_became_active":"2024-09-16T06:19:27.780995+0000","last_became_peered":"2024-09-16T06:19:27.780995+0000","last_unstale":"2024-09-16T06:19:34.252994+0000","last_undegraded":"2024-09-16T06:19:34.252994+0000","last_fullsized":"2024-09-16T06:19:34.252994+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:57.749916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254089+0000","last_change":"2024-09-16T06:19:23.406344+0000","last_active":"2024-09-16T06:19:34.254089+0000","last_peered":"2024-09-16T06:19:34.254089+0000","last_clean":"2024-09-16T06:19:34.254089+0000","last_became_active":"2024-09-16T06:19:23.405994+0000","last_became_peered":"2024-09-16T06:19:23.405994+0000","last_unstale":"2024-09-16T06:19:34.254089+0000","last_undegraded":"2024-09-16T06:19:34.254089+0000","last_fullsized":"2024-09-16T06:19:34.254089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:59.051426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253631+0000","last_change":"2024-09-16T06:19:21.398585+0000","last_active":"2024-09-16T06:19:34.253631+0000","last_peered":"2024-09-16T06:19:34.253631+0000","last_clean":"2024-09-16T06:19:34.253631+0000","last_became_active":"2024-09-16T06:19:21.398252+0000","last_became_peered":"2024-09-16T06:19:21.398252+0000","last_unstale":"2024-09-16T06:19:34.253631+0000","last_undegraded":"2024-09-16T06:19:34.253631+0000","last_fullsized":"2024-09-16T06:19:34.253631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:33.202540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551948+0000","last_change":"2024-09-16T06:19:25.408462+0000","last_active":"2024-09-16T06:20:39.551948+0000","last_peered":"2024-09-16T06:20:39.551948+0000","last_clean":"2024-09-16T06:20:39.551948+0000","last_became_active":"2024-09-16T06:19:25.407924+0000","last_became_peered":"2024-09-16T06:19:25.407924+0000","last_unstale":"2024-09-16T06:20:39.551948+0000","last_undegraded":"2024-09-16T06:20:39.551948+0000","last_fullsized":"2024-09-16T06:20:39.551948+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:17.213277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542216+0000","last_change":"2024-09-16T06:19:27.424241+0000","last_active":"2024-09-16T06:19:34.542216+0000","last_peered":"2024-09-16T06:19:34.542216+0000","last_clean":"2024-09-16T06:19:34.542216+0000","last_became_active":"2024-09-16T06:19:27.423835+0000","last_became_peered":"2024-09-16T06:19:27.423835+0000","last_unstale":"2024-09-16T06:19:34.542216+0000","last_undegraded":"2024-09-16T06:19:34.542216+0000","last_fullsized":"2024-09-16T06:19:34.542216+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:08.709722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":82,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551907+0000","last_change":"2024-09-16T06:19:23.407553+0000","last_active":"2024-09-16T06:20:39.551907+0000","last_peered":"2024-09-16T06:20:39.551907+0000","last_clean":"2024-09-16T06:20:39.551907+0000","last_became_active":"2024-09-16T06:19:23.407282+0000","last_became_peered":"2024-09-16T06:19:23.407282+0000","last_unstale":"2024-09-16T06:20:39.551907+0000","last_undegraded":"2024-09-16T06:20:39.551907+0000","last_fullsized":"2024-09-16T06:20:39.551907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:15.977324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251586+0000","last_change":"2024-09-16T06:19:21.397155+0000","last_active":"2024-09-16T06:19:34.251586+0000","last_peered":"2024-09-16T06:19:34.251586+0000","last_clean":"2024-09-16T06:19:34.251586+0000","last_became_active":"2024-09-16T06:19:21.396313+0000","last_became_peered":"2024-09-16T06:19:21.396313+0000","last_unstale":"2024-09-16T06:19:34.251586+0000","last_undegraded":"2024-09-16T06:19:34.251586+0000","last_fullsized":"2024-09-16T06:19:34.251586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:43.307627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543234+0000","last_change":"2024-09-16T06:19:25.408185+0000","last_active":"2024-09-16T06:19:34.543234+0000","last_peered":"2024-09-16T06:19:34.543234+0000","last_clean":"2024-09-16T06:19:34.543234+0000","last_became_active":"2024-09-16T06:19:25.407885+0000","last_became_peered":"2024-09-16T06:19:25.407885+0000","last_unstale":"2024-09-16T06:19:34.543234+0000","last_undegraded":"2024-09-16T06:19:34.543234+0000","last_fullsized":"2024-09-16T06:19:34.543234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:01.560872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543711+0000","last_change":"2024-09-16T06:19:27.421212+0000","last_active":"2024-09-16T06:19:34.543711+0000","last_peered":"2024-09-16T06:19:34.543711+0000","last_clean":"2024-09-16T06:19:34.543711+0000","last_became_active":"2024-09-16T06:19:27.420929+0000","last_became_peered":"2024-09-16T06:19:27.420929+0000","last_unstale":"2024-09-16T06:19:34.543711+0000","last_undegraded":"2024-09-16T06:19:34.543711+0000","last_fullsized":"2024-09-16T06:19:34.543711+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:22:59.185946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":44,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254157+0000","last_change":"2024-09-16T06:19:23.399957+0000","last_active":"2024-09-16T06:19:34.254157+0000","last_peered":"2024-09-16T06:19:34.254157+0000","last_clean":"2024-09-16T06:19:34.254157+0000","last_became_active":"2024-09-16T06:19:23.399680+0000","last_became_peered":"2024-09-16T06:19:23.399680+0000","last_unstale":"2024-09-16T06:19:34.254157+0000","last_undegraded":"2024-09-16T06:19:34.254157+0000","last_fullsized":"2024-09-16T06:19:34.254157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:12.684835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":104,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:43.517428+0000","last_change":"2024-09-16T06:19:25.406708+0000","last_active":"2024-09-16T06:20:43.517428+0000","last_peered":"2024-09-16T06:20:43.517428+0000","last_clean":"2024-09-16T06:20:43.517428+0000","last_became_active":"2024-09-16T06:19:21.395420+0000","last_became_peered":"2024-09-16T06:19:21.395420+0000","last_unstale":"2024-09-16T06:20:43.517428+0000","last_undegraded":"2024-09-16T06:20:43.517428+0000","last_fullsized":"2024-09-16T06:20:43.517428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:55.321009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068908200000000004,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253611+0000","last_change":"2024-09-16T06:19:21.399103+0000","last_active":"2024-09-16T06:19:34.253611+0000","last_peered":"2024-09-16T06:19:34.253611+0000","last_clean":"2024-09-16T06:19:34.253611+0000","last_became_active":"2024-09-16T06:19:21.398831+0000","last_became_peered":"2024-09-16T06:19:21.398831+0000","last_unstale":"2024-09-16T06:19:34.253611+0000","last_undegraded":"2024-09-16T06:19:34.253611+0000","last_fullsized":"2024-09-16T06:19:34.253611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:03.819465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254012+0000","last_change":"2024-09-16T06:19:25.410642+0000","last_active":"2024-09-16T06:19:34.254012+0000","last_peered":"2024-09-16T06:19:34.254012+0000","last_clean":"2024-09-16T06:19:34.254012+0000","last_became_active":"2024-09-16T06:19:25.410355+0000","last_became_peered":"2024-09-16T06:19:25.410355+0000","last_unstale":"2024-09-16T06:19:34.254012+0000","last_undegraded":"2024-09-16T06:19:34.254012+0000","last_fullsized":"2024-09-16T06:19:34.254012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:43.062687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542688+0000","last_change":"2024-09-16T06:19:27.778671+0000","last_active":"2024-09-16T06:19:34.542688+0000","last_peered":"2024-09-16T06:19:34.542688+0000","last_clean":"2024-09-16T06:19:34.542688+0000","last_became_active":"2024-09-16T06:19:27.778336+0000","last_became_peered":"2024-09-16T06:19:27.778336+0000","last_unstale":"2024-09-16T06:19:34.542688+0000","last_undegraded":"2024-09-16T06:19:34.542688+0000","last_fullsized":"2024-09-16T06:19:34.542688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:54.249497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":35,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543793+0000","last_change":"2024-09-16T06:19:23.398835+0000","last_active":"2024-09-16T06:19:34.543793+0000","last_peered":"2024-09-16T06:19:34.543793+0000","last_clean":"2024-09-16T06:19:34.543793+0000","last_became_active":"2024-09-16T06:19:23.398511+0000","last_became_peered":"2024-09-16T06:19:23.398511+0000","last_unstale":"2024-09-16T06:19:34.543793+0000","last_undegraded":"2024-09-16T06:19:34.543793+0000","last_fullsized":"2024-09-16T06:19:34.543793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:58:36.714926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":25,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543560+0000","last_change":"2024-09-16T06:19:25.397601+0000","last_active":"2024-09-16T06:19:34.543560+0000","last_peered":"2024-09-16T06:19:34.543560+0000","last_clean":"2024-09-16T06:19:34.543560+0000","last_became_active":"2024-09-16T06:19:21.400521+0000","last_became_peered":"2024-09-16T06:19:21.400521+0000","last_unstale":"2024-09-16T06:19:34.543560+0000","last_undegraded":"2024-09-16T06:19:34.543560+0000","last_fullsized":"2024-09-16T06:19:34.543560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:31:56.227673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048356799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541987+0000","last_change":"2024-09-16T06:19:21.396761+0000","last_active":"2024-09-16T06:19:34.541987+0000","last_peered":"2024-09-16T06:19:34.541987+0000","last_clean":"2024-09-16T06:19:34.541987+0000","last_became_active":"2024-09-16T06:19:21.396505+0000","last_became_peered":"2024-09-16T06:19:21.396505+0000","last_unstale":"2024-09-16T06:19:34.541987+0000","last_undegraded":"2024-09-16T06:19:34.541987+0000","last_fullsized":"2024-09-16T06:19:34.541987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:22.462518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252002+0000","last_change":"2024-09-16T06:19:25.413602+0000","last_active":"2024-09-16T06:19:34.252002+0000","last_peered":"2024-09-16T06:19:34.252002+0000","last_clean":"2024-09-16T06:19:34.252002+0000","last_became_active":"2024-09-16T06:19:25.413359+0000","last_became_peered":"2024-09-16T06:19:25.413359+0000","last_unstale":"2024-09-16T06:19:34.252002+0000","last_undegraded":"2024-09-16T06:19:34.252002+0000","last_fullsized":"2024-09-16T06:19:34.252002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:50.911744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253234+0000","last_change":"2024-09-16T06:19:27.426230+0000","last_active":"2024-09-16T06:19:34.253234+0000","last_peered":"2024-09-16T06:19:34.253234+0000","last_clean":"2024-09-16T06:19:34.253234+0000","last_became_active":"2024-09-16T06:19:27.425869+0000","last_became_peered":"2024-09-16T06:19:27.425869+0000","last_unstale":"2024-09-16T06:19:34.253234+0000","last_undegraded":"2024-09-16T06:19:34.253234+0000","last_fullsized":"2024-09-16T06:19:34.253234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:04.155440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'14","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291036+0000","last_change":"2024-09-16T06:19:23.402472+0000","last_active":"2024-09-16T06:19:34.291036+0000","last_peered":"2024-09-16T06:19:34.291036+0000","last_clean":"2024-09-16T06:19:34.291036+0000","last_became_active":"2024-09-16T06:19:23.402013+0000","last_became_peered":"2024-09-16T06:19:23.402013+0000","last_unstale":"2024-09-16T06:19:34.291036+0000","last_undegraded":"2024-09-16T06:19:34.291036+0000","last_fullsized":"2024-09-16T06:19:34.291036+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:10.870975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542899+0000","last_change":"2024-09-16T06:19:25.407357+0000","last_active":"2024-09-16T06:19:34.542899+0000","last_peered":"2024-09-16T06:19:34.542899+0000","last_clean":"2024-09-16T06:19:34.542899+0000","last_became_active":"2024-09-16T06:19:21.399439+0000","last_became_peered":"2024-09-16T06:19:21.399439+0000","last_unstale":"2024-09-16T06:19:34.542899+0000","last_undegraded":"2024-09-16T06:19:34.542899+0000","last_fullsized":"2024-09-16T06:19:34.542899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:52.204830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035565369999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251602+0000","last_change":"2024-09-16T06:19:21.398672+0000","last_active":"2024-09-16T06:19:34.251602+0000","last_peered":"2024-09-16T06:19:34.251602+0000","last_clean":"2024-09-16T06:19:34.251602+0000","last_became_active":"2024-09-16T06:19:21.398171+0000","last_became_peered":"2024-09-16T06:19:21.398171+0000","last_unstale":"2024-09-16T06:19:34.251602+0000","last_undegraded":"2024-09-16T06:19:34.251602+0000","last_fullsized":"2024-09-16T06:19:34.251602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:52.185621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'188","reported_seq":334,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:00.641768+0000","last_change":"2024-09-16T06:19:12.255831+0000","last_active":"2024-09-16T06:20:00.641768+0000","last_peered":"2024-09-16T06:20:00.641768+0000","last_clean":"2024-09-16T06:20:00.641768+0000","last_became_active":"2024-09-16T06:19:12.255346+0000","last_became_peered":"2024-09-16T06:19:12.255346+0000","last_unstale":"2024-09-16T06:20:00.641768+0000","last_undegraded":"2024-09-16T06:20:00.641768+0000","last_fullsized":"2024-09-16T06:20:00.641768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:17:09.239802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:17:09.239802+0000","last_clean_scrub_stamp":"2024-09-16T06:17:09.239802+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:36.608827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542443+0000","last_change":"2024-09-16T06:19:25.413574+0000","last_active":"2024-09-16T06:19:34.542443+0000","last_peered":"2024-09-16T06:19:34.542443+0000","last_clean":"2024-09-16T06:19:34.542443+0000","last_became_active":"2024-09-16T06:19:25.413274+0000","last_became_peered":"2024-09-16T06:19:25.413274+0000","last_unstale":"2024-09-16T06:19:34.542443+0000","last_undegraded":"2024-09-16T06:19:34.542443+0000","last_fullsized":"2024-09-16T06:19:34.542443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:47.539928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252064+0000","last_change":"2024-09-16T06:19:27.422370+0000","last_active":"2024-09-16T06:19:34.252064+0000","last_peered":"2024-09-16T06:19:34.252064+0000","last_clean":"2024-09-16T06:19:34.252064+0000","last_became_active":"2024-09-16T06:19:27.422088+0000","last_became_peered":"2024-09-16T06:19:27.422088+0000","last_unstale":"2024-09-16T06:19:34.252064+0000","last_undegraded":"2024-09-16T06:19:34.252064+0000","last_fullsized":"2024-09-16T06:19:34.252064+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:26.241616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":48,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290923+0000","last_change":"2024-09-16T06:19:23.401999+0000","last_active":"2024-09-16T06:19:34.290923+0000","last_peered":"2024-09-16T06:19:34.290923+0000","last_clean":"2024-09-16T06:19:34.290923+0000","last_became_active":"2024-09-16T06:19:23.401423+0000","last_became_peered":"2024-09-16T06:19:23.401423+0000","last_unstale":"2024-09-16T06:19:34.290923+0000","last_undegraded":"2024-09-16T06:19:34.290923+0000","last_fullsized":"2024-09-16T06:19:34.290923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:26.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254631+0000","last_change":"2024-09-16T06:19:21.402262+0000","last_active":"2024-09-16T06:19:34.254631+0000","last_peered":"2024-09-16T06:19:34.254631+0000","last_clean":"2024-09-16T06:19:34.254631+0000","last_became_active":"2024-09-16T06:19:21.401980+0000","last_became_peered":"2024-09-16T06:19:21.401980+0000","last_unstale":"2024-09-16T06:19:34.254631+0000","last_undegraded":"2024-09-16T06:19:34.254631+0000","last_fullsized":"2024-09-16T06:19:34.254631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:55.910726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253390+0000","last_change":"2024-09-16T06:19:25.405452+0000","last_active":"2024-09-16T06:19:34.253390+0000","last_peered":"2024-09-16T06:19:34.253390+0000","last_clean":"2024-09-16T06:19:34.253390+0000","last_became_active":"2024-09-16T06:19:25.405172+0000","last_became_peered":"2024-09-16T06:19:25.405172+0000","last_unstale":"2024-09-16T06:19:34.253390+0000","last_undegraded":"2024-09-16T06:19:34.253390+0000","last_fullsized":"2024-09-16T06:19:34.253390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:21.097918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542009+0000","last_change":"2024-09-16T06:19:27.423823+0000","last_active":"2024-09-16T06:19:34.542009+0000","last_peered":"2024-09-16T06:19:34.542009+0000","last_clean":"2024-09-16T06:19:34.542009+0000","last_became_active":"2024-09-16T06:19:27.423507+0000","last_became_peered":"2024-09-16T06:19:27.423507+0000","last_unstale":"2024-09-16T06:19:34.542009+0000","last_undegraded":"2024-09-16T06:19:34.542009+0000","last_fullsized":"2024-09-16T06:19:34.542009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:44.696578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543138+0000","last_change":"2024-09-16T06:19:23.402486+0000","last_active":"2024-09-16T06:19:34.543138+0000","last_peered":"2024-09-16T06:19:34.543138+0000","last_clean":"2024-09-16T06:19:34.543138+0000","last_became_active":"2024-09-16T06:19:23.402166+0000","last_became_peered":"2024-09-16T06:19:23.402166+0000","last_unstale":"2024-09-16T06:19:34.543138+0000","last_undegraded":"2024-09-16T06:19:34.543138+0000","last_fullsized":"2024-09-16T06:19:34.543138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:24.452315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253780+0000","last_change":"2024-09-16T06:19:21.398668+0000","last_active":"2024-09-16T06:19:34.253780+0000","last_peered":"2024-09-16T06:19:34.253780+0000","last_clean":"2024-09-16T06:19:34.253780+0000","last_became_active":"2024-09-16T06:19:21.398193+0000","last_became_peered":"2024-09-16T06:19:21.398193+0000","last_unstale":"2024-09-16T06:19:34.253780+0000","last_undegraded":"2024-09-16T06:19:34.253780+0000","last_fullsized":"2024-09-16T06:19:34.253780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.150203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253937+0000","last_change":"2024-09-16T06:19:25.411973+0000","last_active":"2024-09-16T06:19:34.253937+0000","last_peered":"2024-09-16T06:19:34.253937+0000","last_clean":"2024-09-16T06:19:34.253937+0000","last_became_active":"2024-09-16T06:19:25.411709+0000","last_became_peered":"2024-09-16T06:19:25.411709+0000","last_unstale":"2024-09-16T06:19:34.253937+0000","last_undegraded":"2024-09-16T06:19:34.253937+0000","last_fullsized":"2024-09-16T06:19:34.253937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:22.411633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543401+0000","last_change":"2024-09-16T06:19:27.420324+0000","last_active":"2024-09-16T06:19:34.543401+0000","last_peered":"2024-09-16T06:19:34.543401+0000","last_clean":"2024-09-16T06:19:34.543401+0000","last_became_active":"2024-09-16T06:19:27.420009+0000","last_became_peered":"2024-09-16T06:19:27.420009+0000","last_unstale":"2024-09-16T06:19:34.543401+0000","last_undegraded":"2024-09-16T06:19:34.543401+0000","last_fullsized":"2024-09-16T06:19:34.543401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:00.550562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251279+0000","last_change":"2024-09-16T06:19:23.400938+0000","last_active":"2024-09-16T06:19:34.251279+0000","last_peered":"2024-09-16T06:19:34.251279+0000","last_clean":"2024-09-16T06:19:34.251279+0000","last_became_active":"2024-09-16T06:19:23.400531+0000","last_became_peered":"2024-09-16T06:19:23.400531+0000","last_unstale":"2024-09-16T06:19:34.251279+0000","last_undegraded":"2024-09-16T06:19:34.251279+0000","last_fullsized":"2024-09-16T06:19:34.251279+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:38.033621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254225+0000","last_change":"2024-09-16T06:19:21.398464+0000","last_active":"2024-09-16T06:19:34.254225+0000","last_peered":"2024-09-16T06:19:34.254225+0000","last_clean":"2024-09-16T06:19:34.254225+0000","last_became_active":"2024-09-16T06:19:21.398194+0000","last_became_peered":"2024-09-16T06:19:21.398194+0000","last_unstale":"2024-09-16T06:19:34.254225+0000","last_undegraded":"2024-09-16T06:19:34.254225+0000","last_fullsized":"2024-09-16T06:19:34.254225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:33.051658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551831+0000","last_change":"2024-09-16T06:19:25.411530+0000","last_active":"2024-09-16T06:20:39.551831+0000","last_peered":"2024-09-16T06:20:39.551831+0000","last_clean":"2024-09-16T06:20:39.551831+0000","last_became_active":"2024-09-16T06:19:25.411245+0000","last_became_peered":"2024-09-16T06:19:25.411245+0000","last_unstale":"2024-09-16T06:20:39.551831+0000","last_undegraded":"2024-09-16T06:20:39.551831+0000","last_fullsized":"2024-09-16T06:20:39.551831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:09.237659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251361+0000","last_change":"2024-09-16T06:19:27.779842+0000","last_active":"2024-09-16T06:19:34.251361+0000","last_peered":"2024-09-16T06:19:34.251361+0000","last_clean":"2024-09-16T06:19:34.251361+0000","last_became_active":"2024-09-16T06:19:27.779548+0000","last_became_peered":"2024-09-16T06:19:27.779548+0000","last_unstale":"2024-09-16T06:19:34.251361+0000","last_undegraded":"2024-09-16T06:19:34.251361+0000","last_fullsized":"2024-09-16T06:19:34.251361+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:43.686092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541818+0000","last_change":"2024-09-16T06:19:23.410570+0000","last_active":"2024-09-16T06:19:34.541818+0000","last_peered":"2024-09-16T06:19:34.541818+0000","last_clean":"2024-09-16T06:19:34.541818+0000","last_became_active":"2024-09-16T06:19:23.410253+0000","last_became_peered":"2024-09-16T06:19:23.410253+0000","last_unstale":"2024-09-16T06:19:34.541818+0000","last_undegraded":"2024-09-16T06:19:34.541818+0000","last_fullsized":"2024-09-16T06:19:34.541818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:09.015298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542960+0000","last_change":"2024-09-16T06:19:21.394627+0000","last_active":"2024-09-16T06:19:34.542960+0000","last_peered":"2024-09-16T06:19:34.542960+0000","last_clean":"2024-09-16T06:19:34.542960+0000","last_became_active":"2024-09-16T06:19:21.394367+0000","last_became_peered":"2024-09-16T06:19:21.394367+0000","last_unstale":"2024-09-16T06:19:34.542960+0000","last_undegraded":"2024-09-16T06:19:34.542960+0000","last_fullsized":"2024-09-16T06:19:34.542960+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:31.746257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253337+0000","last_change":"2024-09-16T06:19:25.411366+0000","last_active":"2024-09-16T06:19:34.253337+0000","last_peered":"2024-09-16T06:19:34.253337+0000","last_clean":"2024-09-16T06:19:34.253337+0000","last_became_active":"2024-09-16T06:19:25.411081+0000","last_became_peered":"2024-09-16T06:19:25.411081+0000","last_unstale":"2024-09-16T06:19:34.253337+0000","last_undegraded":"2024-09-16T06:19:34.253337+0000","last_fullsized":"2024-09-16T06:19:34.253337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:50.158742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542607+0000","last_change":"2024-09-16T06:19:27.422993+0000","last_active":"2024-09-16T06:19:34.542607+0000","last_peered":"2024-09-16T06:19:34.542607+0000","last_clean":"2024-09-16T06:19:34.542607+0000","last_became_active":"2024-09-16T06:19:27.421604+0000","last_became_peered":"2024-09-16T06:19:27.421604+0000","last_unstale":"2024-09-16T06:19:34.542607+0000","last_undegraded":"2024-09-16T06:19:34.542607+0000","last_fullsized":"2024-09-16T06:19:34.542607+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:55.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543685+0000","last_change":"2024-09-16T06:19:23.404751+0000","last_active":"2024-09-16T06:19:34.543685+0000","last_peered":"2024-09-16T06:19:34.543685+0000","last_clean":"2024-09-16T06:19:34.543685+0000","last_became_active":"2024-09-16T06:19:23.404205+0000","last_became_peered":"2024-09-16T06:19:23.404205+0000","last_unstale":"2024-09-16T06:19:34.543685+0000","last_undegraded":"2024-09-16T06:19:34.543685+0000","last_fullsized":"2024-09-16T06:19:34.543685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:32.881032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253492+0000","last_change":"2024-09-16T06:19:21.402169+0000","last_active":"2024-09-16T06:19:34.253492+0000","last_peered":"2024-09-16T06:19:34.253492+0000","last_clean":"2024-09-16T06:19:34.253492+0000","last_became_active":"2024-09-16T06:19:21.401883+0000","last_became_peered":"2024-09-16T06:19:21.401883+0000","last_unstale":"2024-09-16T06:19:34.253492+0000","last_undegraded":"2024-09-16T06:19:34.253492+0000","last_fullsized":"2024-09-16T06:19:34.253492+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:14.708919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.544025+0000","last_change":"2024-09-16T06:19:25.409659+0000","last_active":"2024-09-16T06:19:34.544025+0000","last_peered":"2024-09-16T06:19:34.544025+0000","last_clean":"2024-09-16T06:19:34.544025+0000","last_became_active":"2024-09-16T06:19:25.409404+0000","last_became_peered":"2024-09-16T06:19:25.409404+0000","last_unstale":"2024-09-16T06:19:34.544025+0000","last_undegraded":"2024-09-16T06:19:34.544025+0000","last_fullsized":"2024-09-16T06:19:34.544025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:05.792174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254139+0000","last_change":"2024-09-16T06:19:27.416443+0000","last_active":"2024-09-16T06:19:34.254139+0000","last_peered":"2024-09-16T06:19:34.254139+0000","last_clean":"2024-09-16T06:19:34.254139+0000","last_became_active":"2024-09-16T06:19:27.416184+0000","last_became_peered":"2024-09-16T06:19:27.416184+0000","last_unstale":"2024-09-16T06:19:34.254139+0000","last_undegraded":"2024-09-16T06:19:34.254139+0000","last_fullsized":"2024-09-16T06:19:34.254139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:46.453311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290840+0000","last_change":"2024-09-16T06:19:21.397729+0000","last_active":"2024-09-16T06:19:34.290840+0000","last_peered":"2024-09-16T06:19:34.290840+0000","last_clean":"2024-09-16T06:19:34.290840+0000","last_became_active":"2024-09-16T06:19:21.397482+0000","last_became_peered":"2024-09-16T06:19:21.397482+0000","last_unstale":"2024-09-16T06:19:34.290840+0000","last_undegraded":"2024-09-16T06:19:34.290840+0000","last_fullsized":"2024-09-16T06:19:34.290840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:48.615614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.544165+0000","last_change":"2024-09-16T06:19:23.408845+0000","last_active":"2024-09-16T06:19:34.544165+0000","last_peered":"2024-09-16T06:19:34.544165+0000","last_clean":"2024-09-16T06:19:34.544165+0000","last_became_active":"2024-09-16T06:19:23.407496+0000","last_became_peered":"2024-09-16T06:19:23.407496+0000","last_unstale":"2024-09-16T06:19:34.544165+0000","last_undegraded":"2024-09-16T06:19:34.544165+0000","last_fullsized":"2024-09-16T06:19:34.544165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:56:41.487842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251933+0000","last_change":"2024-09-16T06:19:25.415353+0000","last_active":"2024-09-16T06:19:34.251933+0000","last_peered":"2024-09-16T06:19:34.251933+0000","last_clean":"2024-09-16T06:19:34.251933+0000","last_became_active":"2024-09-16T06:19:25.415050+0000","last_became_peered":"2024-09-16T06:19:25.415050+0000","last_unstale":"2024-09-16T06:19:34.251933+0000","last_undegraded":"2024-09-16T06:19:34.251933+0000","last_fullsized":"2024-09-16T06:19:34.251933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:19.735099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542619+0000","last_change":"2024-09-16T06:19:27.780194+0000","last_active":"2024-09-16T06:19:34.542619+0000","last_peered":"2024-09-16T06:19:34.542619+0000","last_clean":"2024-09-16T06:19:34.542619+0000","last_became_active":"2024-09-16T06:19:27.779890+0000","last_became_peered":"2024-09-16T06:19:27.779890+0000","last_unstale":"2024-09-16T06:19:34.542619+0000","last_undegraded":"2024-09-16T06:19:34.542619+0000","last_fullsized":"2024-09-16T06:19:34.542619+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:38.244708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543509+0000","last_change":"2024-09-16T06:19:23.411528+0000","last_active":"2024-09-16T06:19:34.543509+0000","last_peered":"2024-09-16T06:19:34.543509+0000","last_clean":"2024-09-16T06:19:34.543509+0000","last_became_active":"2024-09-16T06:19:23.411257+0000","last_became_peered":"2024-09-16T06:19:23.411257+0000","last_unstale":"2024-09-16T06:19:34.543509+0000","last_undegraded":"2024-09-16T06:19:34.543509+0000","last_fullsized":"2024-09-16T06:19:34.543509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:18.438834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251573+0000","last_change":"2024-09-16T06:19:21.398837+0000","last_active":"2024-09-16T06:19:34.251573+0000","last_peered":"2024-09-16T06:19:34.251573+0000","last_clean":"2024-09-16T06:19:34.251573+0000","last_became_active":"2024-09-16T06:19:21.398111+0000","last_became_peered":"2024-09-16T06:19:21.398111+0000","last_unstale":"2024-09-16T06:19:34.251573+0000","last_undegraded":"2024-09-16T06:19:34.251573+0000","last_fullsized":"2024-09-16T06:19:34.251573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:37:07.316318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551837+0000","last_change":"2024-09-16T06:19:25.410242+0000","last_active":"2024-09-16T06:20:39.551837+0000","last_peered":"2024-09-16T06:20:39.551837+0000","last_clean":"2024-09-16T06:20:39.551837+0000","last_became_active":"2024-09-16T06:19:25.409777+0000","last_became_peered":"2024-09-16T06:19:25.409777+0000","last_unstale":"2024-09-16T06:20:39.551837+0000","last_undegraded":"2024-09-16T06:20:39.551837+0000","last_fullsized":"2024-09-16T06:20:39.551837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:15.072946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543370+0000","last_change":"2024-09-16T06:19:27.423715+0000","last_active":"2024-09-16T06:19:34.543370+0000","last_peered":"2024-09-16T06:19:34.543370+0000","last_clean":"2024-09-16T06:19:34.543370+0000","last_became_active":"2024-09-16T06:19:27.423414+0000","last_became_peered":"2024-09-16T06:19:27.423414+0000","last_unstale":"2024-09-16T06:19:34.543370+0000","last_undegraded":"2024-09-16T06:19:34.543370+0000","last_fullsized":"2024-09-16T06:19:34.543370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:42.915520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251891+0000","last_change":"2024-09-16T06:19:23.402973+0000","last_active":"2024-09-16T06:19:34.251891+0000","last_peered":"2024-09-16T06:19:34.251891+0000","last_clean":"2024-09-16T06:19:34.251891+0000","last_became_active":"2024-09-16T06:19:23.402651+0000","last_became_peered":"2024-09-16T06:19:23.402651+0000","last_unstale":"2024-09-16T06:19:34.251891+0000","last_undegraded":"2024-09-16T06:19:34.251891+0000","last_fullsized":"2024-09-16T06:19:34.251891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:28.773138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543271+0000","last_change":"2024-09-16T06:19:21.397191+0000","last_active":"2024-09-16T06:19:34.543271+0000","last_peered":"2024-09-16T06:19:34.543271+0000","last_clean":"2024-09-16T06:19:34.543271+0000","last_became_active":"2024-09-16T06:19:21.396932+0000","last_became_peered":"2024-09-16T06:19:21.396932+0000","last_unstale":"2024-09-16T06:19:34.543271+0000","last_undegraded":"2024-09-16T06:19:34.543271+0000","last_fullsized":"2024-09-16T06:19:34.543271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:19.912733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.552054+0000","last_change":"2024-09-16T06:19:25.408874+0000","last_active":"2024-09-16T06:20:39.552054+0000","last_peered":"2024-09-16T06:20:39.552054+0000","last_clean":"2024-09-16T06:20:39.552054+0000","last_became_active":"2024-09-16T06:19:25.407793+0000","last_became_peered":"2024-09-16T06:19:25.407793+0000","last_unstale":"2024-09-16T06:20:39.552054+0000","last_undegraded":"2024-09-16T06:20:39.552054+0000","last_fullsized":"2024-09-16T06:20:39.552054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:50.017797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542678+0000","last_change":"2024-09-16T06:19:27.416993+0000","last_active":"2024-09-16T06:19:34.542678+0000","last_peered":"2024-09-16T06:19:34.542678+0000","last_clean":"2024-09-16T06:19:34.542678+0000","last_became_active":"2024-09-16T06:19:27.416759+0000","last_became_peered":"2024-09-16T06:19:27.416759+0000","last_unstale":"2024-09-16T06:19:34.542678+0000","last_undegraded":"2024-09-16T06:19:34.542678+0000","last_fullsized":"2024-09-16T06:19:34.542678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:30.611372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254002+0000","last_change":"2024-09-16T06:19:23.401460+0000","last_active":"2024-09-16T06:19:34.254002+0000","last_peered":"2024-09-16T06:19:34.254002+0000","last_clean":"2024-09-16T06:19:34.254002+0000","last_became_active":"2024-09-16T06:19:23.401168+0000","last_became_peered":"2024-09-16T06:19:23.401168+0000","last_unstale":"2024-09-16T06:19:34.254002+0000","last_undegraded":"2024-09-16T06:19:34.254002+0000","last_fullsized":"2024-09-16T06:19:34.254002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:42.903240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291120+0000","last_change":"2024-09-16T06:19:21.405127+0000","last_active":"2024-09-16T06:19:34.291120+0000","last_peered":"2024-09-16T06:19:34.291120+0000","last_clean":"2024-09-16T06:19:34.291120+0000","last_became_active":"2024-09-16T06:19:21.404924+0000","last_became_peered":"2024-09-16T06:19:21.404924+0000","last_unstale":"2024-09-16T06:19:34.291120+0000","last_undegraded":"2024-09-16T06:19:34.291120+0000","last_fullsized":"2024-09-16T06:19:34.291120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:35.406875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542662+0000","last_change":"2024-09-16T06:19:25.410353+0000","last_active":"2024-09-16T06:19:34.542662+0000","last_peered":"2024-09-16T06:19:34.542662+0000","last_clean":"2024-09-16T06:19:34.542662+0000","last_became_active":"2024-09-16T06:19:25.410058+0000","last_became_peered":"2024-09-16T06:19:25.410058+0000","last_unstale":"2024-09-16T06:19:34.542662+0000","last_undegraded":"2024-09-16T06:19:34.542662+0000","last_fullsized":"2024-09-16T06:19:34.542662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:48.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542256+0000","last_change":"2024-09-16T06:19:27.421209+0000","last_active":"2024-09-16T06:19:34.542256+0000","last_peered":"2024-09-16T06:19:34.542256+0000","last_clean":"2024-09-16T06:19:34.542256+0000","last_became_active":"2024-09-16T06:19:27.420885+0000","last_became_peered":"2024-09-16T06:19:27.420885+0000","last_unstale":"2024-09-16T06:19:34.542256+0000","last_undegraded":"2024-09-16T06:19:34.542256+0000","last_fullsized":"2024-09-16T06:19:34.542256+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:09.249936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542247+0000","last_change":"2024-09-16T06:19:23.410647+0000","last_active":"2024-09-16T06:19:34.542247+0000","last_peered":"2024-09-16T06:19:34.542247+0000","last_clean":"2024-09-16T06:19:34.542247+0000","last_became_active":"2024-09-16T06:19:23.410375+0000","last_became_peered":"2024-09-16T06:19:23.410375+0000","last_unstale":"2024-09-16T06:19:34.542247+0000","last_undegraded":"2024-09-16T06:19:34.542247+0000","last_fullsized":"2024-09-16T06:19:34.542247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:59.571144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253682+0000","last_change":"2024-09-16T06:19:21.394709+0000","last_active":"2024-09-16T06:19:34.253682+0000","last_peered":"2024-09-16T06:19:34.253682+0000","last_clean":"2024-09-16T06:19:34.253682+0000","last_became_active":"2024-09-16T06:19:21.393441+0000","last_became_peered":"2024-09-16T06:19:21.393441+0000","last_unstale":"2024-09-16T06:19:34.253682+0000","last_undegraded":"2024-09-16T06:19:34.253682+0000","last_fullsized":"2024-09-16T06:19:34.253682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:19.210905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253835+0000","last_change":"2024-09-16T06:19:25.407913+0000","last_active":"2024-09-16T06:19:34.253835+0000","last_peered":"2024-09-16T06:19:34.253835+0000","last_clean":"2024-09-16T06:19:34.253835+0000","last_became_active":"2024-09-16T06:19:25.407131+0000","last_became_peered":"2024-09-16T06:19:25.407131+0000","last_unstale":"2024-09-16T06:19:34.253835+0000","last_undegraded":"2024-09-16T06:19:34.253835+0000","last_fullsized":"2024-09-16T06:19:34.253835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:42.857972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542269+0000","last_change":"2024-09-16T06:19:27.423373+0000","last_active":"2024-09-16T06:19:34.542269+0000","last_peered":"2024-09-16T06:19:34.542269+0000","last_clean":"2024-09-16T06:19:34.542269+0000","last_became_active":"2024-09-16T06:19:27.423101+0000","last_became_peered":"2024-09-16T06:19:27.423101+0000","last_unstale":"2024-09-16T06:19:34.542269+0000","last_undegraded":"2024-09-16T06:19:34.542269+0000","last_fullsized":"2024-09-16T06:19:34.542269+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:46.996243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542364+0000","last_change":"2024-09-16T06:19:23.407023+0000","last_active":"2024-09-16T06:19:34.542364+0000","last_peered":"2024-09-16T06:19:34.542364+0000","last_clean":"2024-09-16T06:19:34.542364+0000","last_became_active":"2024-09-16T06:19:23.406745+0000","last_became_peered":"2024-09-16T06:19:23.406745+0000","last_unstale":"2024-09-16T06:19:34.542364+0000","last_undegraded":"2024-09-16T06:19:34.542364+0000","last_fullsized":"2024-09-16T06:19:34.542364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.191754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251988+0000","last_change":"2024-09-16T06:19:21.401070+0000","last_active":"2024-09-16T06:19:34.251988+0000","last_peered":"2024-09-16T06:19:34.251988+0000","last_clean":"2024-09-16T06:19:34.251988+0000","last_became_active":"2024-09-16T06:19:21.400799+0000","last_became_peered":"2024-09-16T06:19:21.400799+0000","last_unstale":"2024-09-16T06:19:34.251988+0000","last_undegraded":"2024-09-16T06:19:34.251988+0000","last_fullsized":"2024-09-16T06:19:34.251988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:49.197425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290649+0000","last_change":"2024-09-16T06:19:25.408421+0000","last_active":"2024-09-16T06:19:34.290649+0000","last_peered":"2024-09-16T06:19:34.290649+0000","last_clean":"2024-09-16T06:19:34.290649+0000","last_became_active":"2024-09-16T06:19:25.408097+0000","last_became_peered":"2024-09-16T06:19:25.408097+0000","last_unstale":"2024-09-16T06:19:34.290649+0000","last_undegraded":"2024-09-16T06:19:34.290649+0000","last_fullsized":"2024-09-16T06:19:34.290649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:37:08.841946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541783+0000","last_change":"2024-09-16T06:19:27.415564+0000","last_active":"2024-09-16T06:19:34.541783+0000","last_peered":"2024-09-16T06:19:34.541783+0000","last_clean":"2024-09-16T06:19:34.541783+0000","last_became_active":"2024-09-16T06:19:27.415274+0000","last_became_peered":"2024-09-16T06:19:27.415274+0000","last_unstale":"2024-09-16T06:19:34.541783+0000","last_undegraded":"2024-09-16T06:19:34.541783+0000","last_fullsized":"2024-09-16T06:19:34.541783+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:35.895555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542896+0000","last_change":"2024-09-16T06:19:23.408366+0000","last_active":"2024-09-16T06:19:34.542896+0000","last_peered":"2024-09-16T06:19:34.542896+0000","last_clean":"2024-09-16T06:19:34.542896+0000","last_became_active":"2024-09-16T06:19:23.408005+0000","last_became_peered":"2024-09-16T06:19:23.408005+0000","last_unstale":"2024-09-16T06:19:34.542896+0000","last_undegraded":"2024-09-16T06:19:34.542896+0000","last_fullsized":"2024-09-16T06:19:34.542896+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:40.041095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291190+0000","last_change":"2024-09-16T06:19:21.400031+0000","last_active":"2024-09-16T06:19:34.291190+0000","last_peered":"2024-09-16T06:19:34.291190+0000","last_clean":"2024-09-16T06:19:34.291190+0000","last_became_active":"2024-09-16T06:19:21.398723+0000","last_became_peered":"2024-09-16T06:19:21.398723+0000","last_unstale":"2024-09-16T06:19:34.291190+0000","last_undegraded":"2024-09-16T06:19:34.291190+0000","last_fullsized":"2024-09-16T06:19:34.291190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:34.838879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541669+0000","last_change":"2024-09-16T06:19:25.409291+0000","last_active":"2024-09-16T06:19:34.541669+0000","last_peered":"2024-09-16T06:19:34.541669+0000","last_clean":"2024-09-16T06:19:34.541669+0000","last_became_active":"2024-09-16T06:19:25.408978+0000","last_became_peered":"2024-09-16T06:19:25.408978+0000","last_unstale":"2024-09-16T06:19:34.541669+0000","last_undegraded":"2024-09-16T06:19:34.541669+0000","last_fullsized":"2024-09-16T06:19:34.541669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:09.397670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542568+0000","last_change":"2024-09-16T06:19:27.780703+0000","last_active":"2024-09-16T06:19:34.542568+0000","last_peered":"2024-09-16T06:19:34.542568+0000","last_clean":"2024-09-16T06:19:34.542568+0000","last_became_active":"2024-09-16T06:19:27.780368+0000","last_became_peered":"2024-09-16T06:19:27.780368+0000","last_unstale":"2024-09-16T06:19:34.542568+0000","last_undegraded":"2024-09-16T06:19:34.542568+0000","last_fullsized":"2024-09-16T06:19:34.542568+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:48.322321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542465+0000","last_change":"2024-09-16T06:19:27.418763+0000","last_active":"2024-09-16T06:19:34.542465+0000","last_peered":"2024-09-16T06:19:34.542465+0000","last_clean":"2024-09-16T06:19:34.542465+0000","last_became_active":"2024-09-16T06:19:27.418473+0000","last_became_peered":"2024-09-16T06:19:27.418473+0000","last_unstale":"2024-09-16T06:19:34.542465+0000","last_undegraded":"2024-09-16T06:19:34.542465+0000","last_fullsized":"2024-09-16T06:19:34.542465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:44.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251299+0000","last_change":"2024-09-16T06:19:21.395909+0000","last_active":"2024-09-16T06:19:34.251299+0000","last_peered":"2024-09-16T06:19:34.251299+0000","last_clean":"2024-09-16T06:19:34.251299+0000","last_became_active":"2024-09-16T06:19:21.395653+0000","last_became_peered":"2024-09-16T06:19:21.395653+0000","last_unstale":"2024-09-16T06:19:34.251299+0000","last_undegraded":"2024-09-16T06:19:34.251299+0000","last_fullsized":"2024-09-16T06:19:34.251299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:30.813769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543579+0000","last_change":"2024-09-16T06:19:23.405752+0000","last_active":"2024-09-16T06:19:34.543579+0000","last_peered":"2024-09-16T06:19:34.543579+0000","last_clean":"2024-09-16T06:19:34.543579+0000","last_became_active":"2024-09-16T06:19:23.405364+0000","last_became_peered":"2024-09-16T06:19:23.405364+0000","last_unstale":"2024-09-16T06:19:34.543579+0000","last_undegraded":"2024-09-16T06:19:34.543579+0000","last_fullsized":"2024-09-16T06:19:34.543579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:16.689204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551827+0000","last_change":"2024-09-16T06:19:25.407896+0000","last_active":"2024-09-16T06:20:39.551827+0000","last_peered":"2024-09-16T06:20:39.551827+0000","last_clean":"2024-09-16T06:20:39.551827+0000","last_became_active":"2024-09-16T06:19:25.407591+0000","last_became_peered":"2024-09-16T06:19:25.407591+0000","last_unstale":"2024-09-16T06:20:39.551827+0000","last_undegraded":"2024-09-16T06:20:39.551827+0000","last_fullsized":"2024-09-16T06:20:39.551827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:28.606239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253183+0000","last_change":"2024-09-16T06:19:27.418777+0000","last_active":"2024-09-16T06:19:34.253183+0000","last_peered":"2024-09-16T06:19:34.253183+0000","last_clean":"2024-09-16T06:19:34.253183+0000","last_became_active":"2024-09-16T06:19:27.418444+0000","last_became_peered":"2024-09-16T06:19:27.418444+0000","last_unstale":"2024-09-16T06:19:34.253183+0000","last_undegraded":"2024-09-16T06:19:34.253183+0000","last_fullsized":"2024-09-16T06:19:34.253183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:43.452190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543075+0000","last_change":"2024-09-16T06:19:21.405315+0000","last_active":"2024-09-16T06:19:34.543075+0000","last_peered":"2024-09-16T06:19:34.543075+0000","last_clean":"2024-09-16T06:19:34.543075+0000","last_became_active":"2024-09-16T06:19:21.405046+0000","last_became_peered":"2024-09-16T06:19:21.405046+0000","last_unstale":"2024-09-16T06:19:34.543075+0000","last_undegraded":"2024-09-16T06:19:34.543075+0000","last_fullsized":"2024-09-16T06:19:34.543075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.933552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291091+0000","last_change":"2024-09-16T06:19:23.401764+0000","last_active":"2024-09-16T06:19:34.291091+0000","last_peered":"2024-09-16T06:19:34.291091+0000","last_clean":"2024-09-16T06:19:34.291091+0000","last_became_active":"2024-09-16T06:19:23.401366+0000","last_became_peered":"2024-09-16T06:19:23.401366+0000","last_unstale":"2024-09-16T06:19:34.291091+0000","last_undegraded":"2024-09-16T06:19:34.291091+0000","last_fullsized":"2024-09-16T06:19:34.291091+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:40.312533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543499+0000","last_change":"2024-09-16T06:19:25.408286+0000","last_active":"2024-09-16T06:19:34.543499+0000","last_peered":"2024-09-16T06:19:34.543499+0000","last_clean":"2024-09-16T06:19:34.543499+0000","last_became_active":"2024-09-16T06:19:25.407981+0000","last_became_peered":"2024-09-16T06:19:25.407981+0000","last_unstale":"2024-09-16T06:19:34.543499+0000","last_undegraded":"2024-09-16T06:19:34.543499+0000","last_fullsized":"2024-09-16T06:19:34.543499+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:58.208151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542760+0000","last_change":"2024-09-16T06:19:27.779028+0000","last_active":"2024-09-16T06:19:34.542760+0000","last_peered":"2024-09-16T06:19:34.542760+0000","last_clean":"2024-09-16T06:19:34.542760+0000","last_became_active":"2024-09-16T06:19:27.778712+0000","last_became_peered":"2024-09-16T06:19:27.778712+0000","last_unstale":"2024-09-16T06:19:34.542760+0000","last_undegraded":"2024-09-16T06:19:34.542760+0000","last_fullsized":"2024-09-16T06:19:34.542760+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:49:52.365369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291152+0000","last_change":"2024-09-16T06:19:21.397572+0000","last_active":"2024-09-16T06:19:34.291152+0000","last_peered":"2024-09-16T06:19:34.291152+0000","last_clean":"2024-09-16T06:19:34.291152+0000","last_became_active":"2024-09-16T06:19:21.397293+0000","last_became_peered":"2024-09-16T06:19:21.397293+0000","last_unstale":"2024-09-16T06:19:34.291152+0000","last_undegraded":"2024-09-16T06:19:34.291152+0000","last_fullsized":"2024-09-16T06:19:34.291152+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:24.952651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254429+0000","last_change":"2024-09-16T06:19:23.403386+0000","last_active":"2024-09-16T06:19:34.254429+0000","last_peered":"2024-09-16T06:19:34.254429+0000","last_clean":"2024-09-16T06:19:34.254429+0000","last_became_active":"2024-09-16T06:19:23.403081+0000","last_became_peered":"2024-09-16T06:19:23.403081+0000","last_unstale":"2024-09-16T06:19:34.254429+0000","last_undegraded":"2024-09-16T06:19:34.254429+0000","last_fullsized":"2024-09-16T06:19:34.254429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:18.401286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253562+0000","last_change":"2024-09-16T06:19:25.412344+0000","last_active":"2024-09-16T06:19:34.253562+0000","last_peered":"2024-09-16T06:19:34.253562+0000","last_clean":"2024-09-16T06:19:34.253562+0000","last_became_active":"2024-09-16T06:19:25.412083+0000","last_became_peered":"2024-09-16T06:19:25.412083+0000","last_unstale":"2024-09-16T06:19:34.253562+0000","last_undegraded":"2024-09-16T06:19:34.253562+0000","last_fullsized":"2024-09-16T06:19:34.253562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.639770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543947+0000","last_change":"2024-09-16T06:19:27.778902+0000","last_active":"2024-09-16T06:19:34.543947+0000","last_peered":"2024-09-16T06:19:34.543947+0000","last_clean":"2024-09-16T06:19:34.543947+0000","last_became_active":"2024-09-16T06:19:27.778632+0000","last_became_peered":"2024-09-16T06:19:27.778632+0000","last_unstale":"2024-09-16T06:19:34.543947+0000","last_undegraded":"2024-09-16T06:19:34.543947+0000","last_fullsized":"2024-09-16T06:19:34.543947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:08.990227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543252+0000","last_change":"2024-09-16T06:19:21.398734+0000","last_active":"2024-09-16T06:19:34.543252+0000","last_peered":"2024-09-16T06:19:34.543252+0000","last_clean":"2024-09-16T06:19:34.543252+0000","last_became_active":"2024-09-16T06:19:21.398407+0000","last_became_peered":"2024-09-16T06:19:21.398407+0000","last_unstale":"2024-09-16T06:19:34.543252+0000","last_undegraded":"2024-09-16T06:19:34.543252+0000","last_fullsized":"2024-09-16T06:19:34.543252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:48.189326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543657+0000","last_change":"2024-09-16T06:19:23.408230+0000","last_active":"2024-09-16T06:19:34.543657+0000","last_peered":"2024-09-16T06:19:34.543657+0000","last_clean":"2024-09-16T06:19:34.543657+0000","last_became_active":"2024-09-16T06:19:23.407555+0000","last_became_peered":"2024-09-16T06:19:23.407555+0000","last_unstale":"2024-09-16T06:19:34.543657+0000","last_undegraded":"2024-09-16T06:19:34.543657+0000","last_fullsized":"2024-09-16T06:19:34.543657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:01:06.246213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542263+0000","last_change":"2024-09-16T06:19:25.406475+0000","last_active":"2024-09-16T06:19:34.542263+0000","last_peered":"2024-09-16T06:19:34.542263+0000","last_clean":"2024-09-16T06:19:34.542263+0000","last_became_active":"2024-09-16T06:19:25.404650+0000","last_became_peered":"2024-09-16T06:19:25.404650+0000","last_unstale":"2024-09-16T06:19:34.542263+0000","last_undegraded":"2024-09-16T06:19:34.542263+0000","last_fullsized":"2024-09-16T06:19:34.542263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:31.426984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201301,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28356,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712700,"statfs":{"total":95990841344,"available":95961804800,"internally_reserved":0,"allocated":1372160,"data_stored":911333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58499999999999996}]},{"osd":1,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.52100000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41499999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55800000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32428,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708628,"statfs":{"total":95990841344,"available":95957635072,"internally_reserved":0,"allocated":1343488,"data_stored":909220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.45700000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.311}]},{"osd":7,"last update":"Mon Sep 16 06:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.61099999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713296,"statfs":{"total":95990841344,"available":95962415104,"internally_reserved":0,"allocated":757760,"data_stored":319384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.34000000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59699999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.498},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48499999999999999},"min":{"1min":0.36699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.46800000000000003,"15min":0.46100000000000002},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63600000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.40600000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.38300000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49299999999999999},"min":{"1min":0.39300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36799999999999999,"15min":0.36499999999999999},"min":{"1min":0.27000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.32700000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.434}]},{"osd":6,"last update":"Mon Sep 16 06:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713268,"statfs":{"total":95990841344,"available":95962386432,"internally_reserved":0,"allocated":786432,"data_stored":320740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.752,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.502}]},{"osd":1,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.49099999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.29199999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29999999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59999999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40899999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39100000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.373},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47499999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31892,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709164,"statfs":{"total":95990841344,"available":95958183936,"internally_reserved":0,"allocated":794624,"data_stored":319944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.38200000000000001,"15min":0.40500000000000003},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.40100000000000002},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.50900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.29799999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47499999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.25900000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.434,"15min":0.44800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.53700000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.375},{"interface":"front","average":{"1min":0.37,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.46600000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.311}]},{"osd":4,"last update":"Mon Sep 16 06:19:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.443},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.749,"5min":0.755,"15min":0.755},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.496}]},{"osd":6,"last update":"Mon Sep 16 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713292,"statfs":{"total":95990841344,"available":95962411008,"internally_reserved":0,"allocated":761856,"data_stored":319856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.25}]},{"osd":1,"last update":"Mon Sep 16 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.42499999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.46999999999999997}]},{"osd":3,"last update":"Mon Sep 16 06:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40500000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.39500000000000002,"15min":0.40899999999999997},"min":{"1min":0.25900000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:19:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68000000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.53400000000000003}]},{"osd":5,"last update":"Mon Sep 16 06:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66900000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.438}]},{"osd":6,"last update":"Mon Sep 16 06:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45600000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.621},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713240,"statfs":{"total":95990841344,"available":95962357760,"internally_reserved":0,"allocated":815104,"data_stored":321434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.318},"min":{"1min":0.23499999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.32100000000000001},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.38500000000000001,"15min":0.38100000000000001},"min":{"1min":0.27800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Sep 16 06:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Sep 16 06:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.5}]},{"osd":5,"last update":"Mon Sep 16 06:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.623}]},{"osd":7,"last update":"Mon Sep 16 06:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28360,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712696,"statfs":{"total":95990841344,"available":95961800704,"internally_reserved":0,"allocated":1376256,"data_stored":911361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.30399999999999999},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.246},{"interface":"front","average":{"1min":0.373,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52100000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36599999999999999,"15min":0.36199999999999999},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.379,"15min":0.371},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.439},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.36299999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:19:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80400000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.71499999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.439},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40200000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:20:45.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T06:20:45.622 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:45 smithi196 ceph-mon[30578]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:45.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:45 smithi196 ceph-mon[30578]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:20:46.014 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:45 smithi190 ceph-mon[25888]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:46.015 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:45 smithi190 ceph-mon[25888]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:20:46.015 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:45 smithi190 ceph-mon[39731]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:46.015 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:45 smithi190 ceph-mon[39731]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:20:46.015 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:20:45 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:20:45.734Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002940102s 2024-09-16T06:20:46.490 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:46 smithi196 podman[71626]: 2024-09-16 06:20:46.363703936 +0000 UTC m=+0.021042893 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-16T06:20:46.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:46 smithi196 podman[71626]: 2024-09-16 06:20:46.584700542 +0000 UTC m=+0.242039497 container create 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, com.redhat.component=ubi8-container, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., release=239.1651231664, build-date=2022-04-29T11:48:11.055897, version=8.5, io.buildah.version=1.32.0, maintainer=Paul Cuzner , architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-16T06:20:47.451 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:20:47.453 INFO:teuthology.orchestra.run.smithi190.stderr:dumped all 2024-09-16T06:20:47.750 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:47 smithi190 ceph-mon[25888]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:47.750 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:47 smithi190 ceph-mon[39731]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:48.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:47 smithi196 ceph-mon[30578]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:48.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:47 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:20:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:20:48.224 INFO:teuthology.orchestra.run.smithi190.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T06:20:46.595484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1029,"num_read_kb":872,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":232164,"kb_used_data":7820,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":749696284,"statfs":{"total":767926730752,"available":767688994816,"internally_reserved":0,"allocated":8007680,"data_stored":4333272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003088"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542507+0000","last_change":"2024-09-16T06:19:27.781523+0000","last_active":"2024-09-16T06:19:34.542507+0000","last_peered":"2024-09-16T06:19:34.542507+0000","last_clean":"2024-09-16T06:19:34.542507+0000","last_became_active":"2024-09-16T06:19:27.780398+0000","last_became_peered":"2024-09-16T06:19:27.780398+0000","last_unstale":"2024-09-16T06:19:34.542507+0000","last_undegraded":"2024-09-16T06:19:34.542507+0000","last_fullsized":"2024-09-16T06:19:34.542507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:34.851192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.544026+0000","last_change":"2024-09-16T06:19:21.393343+0000","last_active":"2024-09-16T06:19:34.544026+0000","last_peered":"2024-09-16T06:19:34.544026+0000","last_clean":"2024-09-16T06:19:34.544026+0000","last_became_active":"2024-09-16T06:19:21.393038+0000","last_became_peered":"2024-09-16T06:19:21.393038+0000","last_unstale":"2024-09-16T06:19:34.544026+0000","last_undegraded":"2024-09-16T06:19:34.544026+0000","last_fullsized":"2024-09-16T06:19:34.544026+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:50.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542899+0000","last_change":"2024-09-16T06:19:23.408766+0000","last_active":"2024-09-16T06:19:34.542899+0000","last_peered":"2024-09-16T06:19:34.542899+0000","last_clean":"2024-09-16T06:19:34.542899+0000","last_became_active":"2024-09-16T06:19:23.408446+0000","last_became_peered":"2024-09-16T06:19:23.408446+0000","last_unstale":"2024-09-16T06:19:34.542899+0000","last_undegraded":"2024-09-16T06:19:34.542899+0000","last_fullsized":"2024-09-16T06:19:34.542899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:23.425549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541740+0000","last_change":"2024-09-16T06:19:25.409178+0000","last_active":"2024-09-16T06:19:34.541740+0000","last_peered":"2024-09-16T06:19:34.541740+0000","last_clean":"2024-09-16T06:19:34.541740+0000","last_became_active":"2024-09-16T06:19:25.408903+0000","last_became_peered":"2024-09-16T06:19:25.408903+0000","last_unstale":"2024-09-16T06:19:34.541740+0000","last_undegraded":"2024-09-16T06:19:34.541740+0000","last_fullsized":"2024-09-16T06:19:34.541740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:30.736315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543873+0000","last_change":"2024-09-16T06:19:21.405390+0000","last_active":"2024-09-16T06:19:34.543873+0000","last_peered":"2024-09-16T06:19:34.543873+0000","last_clean":"2024-09-16T06:19:34.543873+0000","last_became_active":"2024-09-16T06:19:21.405007+0000","last_became_peered":"2024-09-16T06:19:21.405007+0000","last_unstale":"2024-09-16T06:19:34.543873+0000","last_undegraded":"2024-09-16T06:19:34.543873+0000","last_fullsized":"2024-09-16T06:19:34.543873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:56.931842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541752+0000","last_change":"2024-09-16T06:19:23.406457+0000","last_active":"2024-09-16T06:19:34.541752+0000","last_peered":"2024-09-16T06:19:34.541752+0000","last_clean":"2024-09-16T06:19:34.541752+0000","last_became_active":"2024-09-16T06:19:23.406129+0000","last_became_peered":"2024-09-16T06:19:23.406129+0000","last_unstale":"2024-09-16T06:19:34.541752+0000","last_undegraded":"2024-09-16T06:19:34.541752+0000","last_fullsized":"2024-09-16T06:19:34.541752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:39.561282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253043+0000","last_change":"2024-09-16T06:19:25.409113+0000","last_active":"2024-09-16T06:19:34.253043+0000","last_peered":"2024-09-16T06:19:34.253043+0000","last_clean":"2024-09-16T06:19:34.253043+0000","last_became_active":"2024-09-16T06:19:25.408801+0000","last_became_peered":"2024-09-16T06:19:25.408801+0000","last_unstale":"2024-09-16T06:19:34.253043+0000","last_undegraded":"2024-09-16T06:19:34.253043+0000","last_fullsized":"2024-09-16T06:19:34.253043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:09.045956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542235+0000","last_change":"2024-09-16T06:19:27.417778+0000","last_active":"2024-09-16T06:19:34.542235+0000","last_peered":"2024-09-16T06:19:34.542235+0000","last_clean":"2024-09-16T06:19:34.542235+0000","last_became_active":"2024-09-16T06:19:27.417541+0000","last_became_peered":"2024-09-16T06:19:27.417541+0000","last_unstale":"2024-09-16T06:19:34.542235+0000","last_undegraded":"2024-09-16T06:19:34.542235+0000","last_fullsized":"2024-09-16T06:19:34.542235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:34.254452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542924+0000","last_change":"2024-09-16T06:19:21.394356+0000","last_active":"2024-09-16T06:19:34.542924+0000","last_peered":"2024-09-16T06:19:34.542924+0000","last_clean":"2024-09-16T06:19:34.542924+0000","last_became_active":"2024-09-16T06:19:21.394112+0000","last_became_peered":"2024-09-16T06:19:21.394112+0000","last_unstale":"2024-09-16T06:19:34.542924+0000","last_undegraded":"2024-09-16T06:19:34.542924+0000","last_fullsized":"2024-09-16T06:19:34.542924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:07:12.607205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542163+0000","last_change":"2024-09-16T06:19:23.410576+0000","last_active":"2024-09-16T06:19:34.542163+0000","last_peered":"2024-09-16T06:19:34.542163+0000","last_clean":"2024-09-16T06:19:34.542163+0000","last_became_active":"2024-09-16T06:19:23.410299+0000","last_became_peered":"2024-09-16T06:19:23.410299+0000","last_unstale":"2024-09-16T06:19:34.542163+0000","last_undegraded":"2024-09-16T06:19:34.542163+0000","last_fullsized":"2024-09-16T06:19:34.542163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:46.031823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542208+0000","last_change":"2024-09-16T06:19:25.409592+0000","last_active":"2024-09-16T06:19:34.542208+0000","last_peered":"2024-09-16T06:19:34.542208+0000","last_clean":"2024-09-16T06:19:34.542208+0000","last_became_active":"2024-09-16T06:19:25.409296+0000","last_became_peered":"2024-09-16T06:19:25.409296+0000","last_unstale":"2024-09-16T06:19:34.542208+0000","last_undegraded":"2024-09-16T06:19:34.542208+0000","last_fullsized":"2024-09-16T06:19:34.542208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:25.391296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251366+0000","last_change":"2024-09-16T06:19:27.421807+0000","last_active":"2024-09-16T06:19:34.251366+0000","last_peered":"2024-09-16T06:19:34.251366+0000","last_clean":"2024-09-16T06:19:34.251366+0000","last_became_active":"2024-09-16T06:19:27.421512+0000","last_became_peered":"2024-09-16T06:19:27.421512+0000","last_unstale":"2024-09-16T06:19:34.251366+0000","last_undegraded":"2024-09-16T06:19:34.251366+0000","last_fullsized":"2024-09-16T06:19:34.251366+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:53.714361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542666+0000","last_change":"2024-09-16T06:19:21.393902+0000","last_active":"2024-09-16T06:19:34.542666+0000","last_peered":"2024-09-16T06:19:34.542666+0000","last_clean":"2024-09-16T06:19:34.542666+0000","last_became_active":"2024-09-16T06:19:21.393626+0000","last_became_peered":"2024-09-16T06:19:21.393626+0000","last_unstale":"2024-09-16T06:19:34.542666+0000","last_undegraded":"2024-09-16T06:19:34.542666+0000","last_fullsized":"2024-09-16T06:19:34.542666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:09.812591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542473+0000","last_change":"2024-09-16T06:19:23.404483+0000","last_active":"2024-09-16T06:19:34.542473+0000","last_peered":"2024-09-16T06:19:34.542473+0000","last_clean":"2024-09-16T06:19:34.542473+0000","last_became_active":"2024-09-16T06:19:23.403509+0000","last_became_peered":"2024-09-16T06:19:23.403509+0000","last_unstale":"2024-09-16T06:19:34.542473+0000","last_undegraded":"2024-09-16T06:19:34.542473+0000","last_fullsized":"2024-09-16T06:19:34.542473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:37:41.966253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251488+0000","last_change":"2024-09-16T06:19:25.406805+0000","last_active":"2024-09-16T06:19:34.251488+0000","last_peered":"2024-09-16T06:19:34.251488+0000","last_clean":"2024-09-16T06:19:34.251488+0000","last_became_active":"2024-09-16T06:19:25.406523+0000","last_became_peered":"2024-09-16T06:19:25.406523+0000","last_unstale":"2024-09-16T06:19:34.251488+0000","last_undegraded":"2024-09-16T06:19:34.251488+0000","last_fullsized":"2024-09-16T06:19:34.251488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:41.291727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542358+0000","last_change":"2024-09-16T06:19:27.420331+0000","last_active":"2024-09-16T06:19:34.542358+0000","last_peered":"2024-09-16T06:19:34.542358+0000","last_clean":"2024-09-16T06:19:34.542358+0000","last_became_active":"2024-09-16T06:19:27.420052+0000","last_became_peered":"2024-09-16T06:19:27.420052+0000","last_unstale":"2024-09-16T06:19:34.542358+0000","last_undegraded":"2024-09-16T06:19:34.542358+0000","last_fullsized":"2024-09-16T06:19:34.542358+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:03.556292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":45,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542759+0000","last_change":"2024-09-16T06:19:23.409283+0000","last_active":"2024-09-16T06:19:34.542759+0000","last_peered":"2024-09-16T06:19:34.542759+0000","last_clean":"2024-09-16T06:19:34.542759+0000","last_became_active":"2024-09-16T06:19:23.408938+0000","last_became_peered":"2024-09-16T06:19:23.408938+0000","last_unstale":"2024-09-16T06:19:34.542759+0000","last_undegraded":"2024-09-16T06:19:34.542759+0000","last_fullsized":"2024-09-16T06:19:34.542759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:59:47.089761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542297+0000","last_change":"2024-09-16T06:19:21.399036+0000","last_active":"2024-09-16T06:19:34.542297+0000","last_peered":"2024-09-16T06:19:34.542297+0000","last_clean":"2024-09-16T06:19:34.542297+0000","last_became_active":"2024-09-16T06:19:21.398661+0000","last_became_peered":"2024-09-16T06:19:21.398661+0000","last_unstale":"2024-09-16T06:19:34.542297+0000","last_undegraded":"2024-09-16T06:19:34.542297+0000","last_fullsized":"2024-09-16T06:19:34.542297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:54.383899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253327+0000","last_change":"2024-09-16T06:19:25.409527+0000","last_active":"2024-09-16T06:19:34.253327+0000","last_peered":"2024-09-16T06:19:34.253327+0000","last_clean":"2024-09-16T06:19:34.253327+0000","last_became_active":"2024-09-16T06:19:25.409148+0000","last_became_peered":"2024-09-16T06:19:25.409148+0000","last_unstale":"2024-09-16T06:19:34.253327+0000","last_undegraded":"2024-09-16T06:19:34.253327+0000","last_fullsized":"2024-09-16T06:19:34.253327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:13:52.863638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254398+0000","last_change":"2024-09-16T06:19:27.426258+0000","last_active":"2024-09-16T06:19:34.254398+0000","last_peered":"2024-09-16T06:19:34.254398+0000","last_clean":"2024-09-16T06:19:34.254398+0000","last_became_active":"2024-09-16T06:19:27.425948+0000","last_became_peered":"2024-09-16T06:19:27.425948+0000","last_unstale":"2024-09-16T06:19:34.254398+0000","last_undegraded":"2024-09-16T06:19:34.254398+0000","last_fullsized":"2024-09-16T06:19:34.254398+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:36.470944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542362+0000","last_change":"2024-09-16T06:19:23.405156+0000","last_active":"2024-09-16T06:19:34.542362+0000","last_peered":"2024-09-16T06:19:34.542362+0000","last_clean":"2024-09-16T06:19:34.542362+0000","last_became_active":"2024-09-16T06:19:23.404618+0000","last_became_peered":"2024-09-16T06:19:23.404618+0000","last_unstale":"2024-09-16T06:19:34.542362+0000","last_undegraded":"2024-09-16T06:19:34.542362+0000","last_fullsized":"2024-09-16T06:19:34.542362+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:09.897750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254206+0000","last_change":"2024-09-16T06:19:21.399408+0000","last_active":"2024-09-16T06:19:34.254206+0000","last_peered":"2024-09-16T06:19:34.254206+0000","last_clean":"2024-09-16T06:19:34.254206+0000","last_became_active":"2024-09-16T06:19:21.399124+0000","last_became_peered":"2024-09-16T06:19:21.399124+0000","last_unstale":"2024-09-16T06:19:34.254206+0000","last_undegraded":"2024-09-16T06:19:34.254206+0000","last_fullsized":"2024-09-16T06:19:34.254206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:18.577960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551937+0000","last_change":"2024-09-16T06:19:25.406608+0000","last_active":"2024-09-16T06:20:39.551937+0000","last_peered":"2024-09-16T06:20:39.551937+0000","last_clean":"2024-09-16T06:20:39.551937+0000","last_became_active":"2024-09-16T06:19:25.406313+0000","last_became_peered":"2024-09-16T06:19:25.406313+0000","last_unstale":"2024-09-16T06:20:39.551937+0000","last_undegraded":"2024-09-16T06:20:39.551937+0000","last_fullsized":"2024-09-16T06:20:39.551937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:14.545971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542694+0000","last_change":"2024-09-16T06:19:27.413145+0000","last_active":"2024-09-16T06:19:34.542694+0000","last_peered":"2024-09-16T06:19:34.542694+0000","last_clean":"2024-09-16T06:19:34.542694+0000","last_became_active":"2024-09-16T06:19:27.412859+0000","last_became_peered":"2024-09-16T06:19:27.412859+0000","last_unstale":"2024-09-16T06:19:34.542694+0000","last_undegraded":"2024-09-16T06:19:34.542694+0000","last_fullsized":"2024-09-16T06:19:34.542694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:47:57.903427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254074+0000","last_change":"2024-09-16T06:19:23.406354+0000","last_active":"2024-09-16T06:19:34.254074+0000","last_peered":"2024-09-16T06:19:34.254074+0000","last_clean":"2024-09-16T06:19:34.254074+0000","last_became_active":"2024-09-16T06:19:23.405896+0000","last_became_peered":"2024-09-16T06:19:23.405896+0000","last_unstale":"2024-09-16T06:19:34.254074+0000","last_undegraded":"2024-09-16T06:19:34.254074+0000","last_fullsized":"2024-09-16T06:19:34.254074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:20.918608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253858+0000","last_change":"2024-09-16T06:19:21.400988+0000","last_active":"2024-09-16T06:19:34.253858+0000","last_peered":"2024-09-16T06:19:34.253858+0000","last_clean":"2024-09-16T06:19:34.253858+0000","last_became_active":"2024-09-16T06:19:21.400095+0000","last_became_peered":"2024-09-16T06:19:21.400095+0000","last_unstale":"2024-09-16T06:19:34.253858+0000","last_undegraded":"2024-09-16T06:19:34.253858+0000","last_fullsized":"2024-09-16T06:19:34.253858+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:59.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551898+0000","last_change":"2024-09-16T06:19:25.406216+0000","last_active":"2024-09-16T06:20:39.551898+0000","last_peered":"2024-09-16T06:20:39.551898+0000","last_clean":"2024-09-16T06:20:39.551898+0000","last_became_active":"2024-09-16T06:19:25.405934+0000","last_became_peered":"2024-09-16T06:19:25.405934+0000","last_unstale":"2024-09-16T06:20:39.551898+0000","last_undegraded":"2024-09-16T06:20:39.551898+0000","last_fullsized":"2024-09-16T06:20:39.551898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:30:43.587414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251715+0000","last_change":"2024-09-16T06:19:27.420455+0000","last_active":"2024-09-16T06:19:34.251715+0000","last_peered":"2024-09-16T06:19:34.251715+0000","last_clean":"2024-09-16T06:19:34.251715+0000","last_became_active":"2024-09-16T06:19:27.420216+0000","last_became_peered":"2024-09-16T06:19:27.420216+0000","last_unstale":"2024-09-16T06:19:34.251715+0000","last_undegraded":"2024-09-16T06:19:34.251715+0000","last_fullsized":"2024-09-16T06:19:34.251715+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:00.104650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542928+0000","last_change":"2024-09-16T06:19:23.410684+0000","last_active":"2024-09-16T06:19:34.542928+0000","last_peered":"2024-09-16T06:19:34.542928+0000","last_clean":"2024-09-16T06:19:34.542928+0000","last_became_active":"2024-09-16T06:19:23.410359+0000","last_became_peered":"2024-09-16T06:19:23.410359+0000","last_unstale":"2024-09-16T06:19:34.542928+0000","last_undegraded":"2024-09-16T06:19:34.542928+0000","last_fullsized":"2024-09-16T06:19:34.542928+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:59.730154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541931+0000","last_change":"2024-09-16T06:19:21.396555+0000","last_active":"2024-09-16T06:19:34.541931+0000","last_peered":"2024-09-16T06:19:34.541931+0000","last_clean":"2024-09-16T06:19:34.541931+0000","last_became_active":"2024-09-16T06:19:21.396224+0000","last_became_peered":"2024-09-16T06:19:21.396224+0000","last_unstale":"2024-09-16T06:19:34.541931+0000","last_undegraded":"2024-09-16T06:19:34.541931+0000","last_fullsized":"2024-09-16T06:19:34.541931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:28.462437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252002+0000","last_change":"2024-09-16T06:19:25.413057+0000","last_active":"2024-09-16T06:19:34.252002+0000","last_peered":"2024-09-16T06:19:34.252002+0000","last_clean":"2024-09-16T06:19:34.252002+0000","last_became_active":"2024-09-16T06:19:25.412711+0000","last_became_peered":"2024-09-16T06:19:25.412711+0000","last_unstale":"2024-09-16T06:19:34.252002+0000","last_undegraded":"2024-09-16T06:19:34.252002+0000","last_fullsized":"2024-09-16T06:19:34.252002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:10.860914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543126+0000","last_change":"2024-09-16T06:19:27.778907+0000","last_active":"2024-09-16T06:19:34.543126+0000","last_peered":"2024-09-16T06:19:34.543126+0000","last_clean":"2024-09-16T06:19:34.543126+0000","last_became_active":"2024-09-16T06:19:27.778591+0000","last_became_peered":"2024-09-16T06:19:27.778591+0000","last_unstale":"2024-09-16T06:19:34.543126+0000","last_undegraded":"2024-09-16T06:19:34.543126+0000","last_fullsized":"2024-09-16T06:19:34.543126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:29.207879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541926+0000","last_change":"2024-09-16T06:19:23.402100+0000","last_active":"2024-09-16T06:19:34.541926+0000","last_peered":"2024-09-16T06:19:34.541926+0000","last_clean":"2024-09-16T06:19:34.541926+0000","last_became_active":"2024-09-16T06:19:23.401840+0000","last_became_peered":"2024-09-16T06:19:23.401840+0000","last_unstale":"2024-09-16T06:19:34.541926+0000","last_undegraded":"2024-09-16T06:19:34.541926+0000","last_fullsized":"2024-09-16T06:19:34.541926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:04.039428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291207+0000","last_change":"2024-09-16T06:19:21.394696+0000","last_active":"2024-09-16T06:19:34.291207+0000","last_peered":"2024-09-16T06:19:34.291207+0000","last_clean":"2024-09-16T06:19:34.291207+0000","last_became_active":"2024-09-16T06:19:21.393723+0000","last_became_peered":"2024-09-16T06:19:21.393723+0000","last_unstale":"2024-09-16T06:19:34.291207+0000","last_undegraded":"2024-09-16T06:19:34.291207+0000","last_fullsized":"2024-09-16T06:19:34.291207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:24.190761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551960+0000","last_change":"2024-09-16T06:19:25.411198+0000","last_active":"2024-09-16T06:20:39.551960+0000","last_peered":"2024-09-16T06:20:39.551960+0000","last_clean":"2024-09-16T06:20:39.551960+0000","last_became_active":"2024-09-16T06:19:25.410886+0000","last_became_peered":"2024-09-16T06:19:25.410886+0000","last_unstale":"2024-09-16T06:20:39.551960+0000","last_undegraded":"2024-09-16T06:20:39.551960+0000","last_fullsized":"2024-09-16T06:20:39.551960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:53.964645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542112+0000","last_change":"2024-09-16T06:19:27.780730+0000","last_active":"2024-09-16T06:19:34.542112+0000","last_peered":"2024-09-16T06:19:34.542112+0000","last_clean":"2024-09-16T06:19:34.542112+0000","last_became_active":"2024-09-16T06:19:27.780369+0000","last_became_peered":"2024-09-16T06:19:27.780369+0000","last_unstale":"2024-09-16T06:19:34.542112+0000","last_undegraded":"2024-09-16T06:19:34.542112+0000","last_fullsized":"2024-09-16T06:19:34.542112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.083603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":49,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542428+0000","last_change":"2024-09-16T06:19:23.403324+0000","last_active":"2024-09-16T06:19:34.542428+0000","last_peered":"2024-09-16T06:19:34.542428+0000","last_clean":"2024-09-16T06:19:34.542428+0000","last_became_active":"2024-09-16T06:19:23.402993+0000","last_became_peered":"2024-09-16T06:19:23.402993+0000","last_unstale":"2024-09-16T06:19:34.542428+0000","last_undegraded":"2024-09-16T06:19:34.542428+0000","last_fullsized":"2024-09-16T06:19:34.542428+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:22:22.218412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253653+0000","last_change":"2024-09-16T06:19:21.395578+0000","last_active":"2024-09-16T06:19:34.253653+0000","last_peered":"2024-09-16T06:19:34.253653+0000","last_clean":"2024-09-16T06:19:34.253653+0000","last_became_active":"2024-09-16T06:19:21.395300+0000","last_became_peered":"2024-09-16T06:19:21.395300+0000","last_unstale":"2024-09-16T06:19:34.253653+0000","last_undegraded":"2024-09-16T06:19:34.253653+0000","last_fullsized":"2024-09-16T06:19:34.253653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:05.210405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541814+0000","last_change":"2024-09-16T06:19:25.407007+0000","last_active":"2024-09-16T06:19:34.541814+0000","last_peered":"2024-09-16T06:19:34.541814+0000","last_clean":"2024-09-16T06:19:34.541814+0000","last_became_active":"2024-09-16T06:19:25.406718+0000","last_became_peered":"2024-09-16T06:19:25.406718+0000","last_unstale":"2024-09-16T06:19:34.541814+0000","last_undegraded":"2024-09-16T06:19:34.541814+0000","last_fullsized":"2024-09-16T06:19:34.541814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:25.088858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542433+0000","last_change":"2024-09-16T06:19:27.423447+0000","last_active":"2024-09-16T06:19:34.542433+0000","last_peered":"2024-09-16T06:19:34.542433+0000","last_clean":"2024-09-16T06:19:34.542433+0000","last_became_active":"2024-09-16T06:19:27.423157+0000","last_became_peered":"2024-09-16T06:19:27.423157+0000","last_unstale":"2024-09-16T06:19:34.542433+0000","last_undegraded":"2024-09-16T06:19:34.542433+0000","last_fullsized":"2024-09-16T06:19:34.542433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:46.280210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":53,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543191+0000","last_change":"2024-09-16T06:19:23.405382+0000","last_active":"2024-09-16T06:19:34.543191+0000","last_peered":"2024-09-16T06:19:34.543191+0000","last_clean":"2024-09-16T06:19:34.543191+0000","last_became_active":"2024-09-16T06:19:23.404131+0000","last_became_peered":"2024-09-16T06:19:23.404131+0000","last_unstale":"2024-09-16T06:19:34.543191+0000","last_undegraded":"2024-09-16T06:19:34.543191+0000","last_fullsized":"2024-09-16T06:19:34.543191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:17.080529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"61'1","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542994+0000","last_change":"2024-09-16T06:19:21.395826+0000","last_active":"2024-09-16T06:19:34.542994+0000","last_peered":"2024-09-16T06:19:34.542994+0000","last_clean":"2024-09-16T06:19:34.542994+0000","last_became_active":"2024-09-16T06:19:21.395569+0000","last_became_peered":"2024-09-16T06:19:21.395569+0000","last_unstale":"2024-09-16T06:19:34.542994+0000","last_undegraded":"2024-09-16T06:19:34.542994+0000","last_fullsized":"2024-09-16T06:19:34.542994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:44.665552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290788+0000","last_change":"2024-09-16T06:19:25.408281+0000","last_active":"2024-09-16T06:19:34.290788+0000","last_peered":"2024-09-16T06:19:34.290788+0000","last_clean":"2024-09-16T06:19:34.290788+0000","last_became_active":"2024-09-16T06:19:25.407940+0000","last_became_peered":"2024-09-16T06:19:25.407940+0000","last_unstale":"2024-09-16T06:19:34.290788+0000","last_undegraded":"2024-09-16T06:19:34.290788+0000","last_fullsized":"2024-09-16T06:19:34.290788+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:55.862327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252994+0000","last_change":"2024-09-16T06:19:27.781363+0000","last_active":"2024-09-16T06:19:34.252994+0000","last_peered":"2024-09-16T06:19:34.252994+0000","last_clean":"2024-09-16T06:19:34.252994+0000","last_became_active":"2024-09-16T06:19:27.780995+0000","last_became_peered":"2024-09-16T06:19:27.780995+0000","last_unstale":"2024-09-16T06:19:34.252994+0000","last_undegraded":"2024-09-16T06:19:34.252994+0000","last_fullsized":"2024-09-16T06:19:34.252994+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:57.749916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254089+0000","last_change":"2024-09-16T06:19:23.406344+0000","last_active":"2024-09-16T06:19:34.254089+0000","last_peered":"2024-09-16T06:19:34.254089+0000","last_clean":"2024-09-16T06:19:34.254089+0000","last_became_active":"2024-09-16T06:19:23.405994+0000","last_became_peered":"2024-09-16T06:19:23.405994+0000","last_unstale":"2024-09-16T06:19:34.254089+0000","last_undegraded":"2024-09-16T06:19:34.254089+0000","last_fullsized":"2024-09-16T06:19:34.254089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:59.051426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253631+0000","last_change":"2024-09-16T06:19:21.398585+0000","last_active":"2024-09-16T06:19:34.253631+0000","last_peered":"2024-09-16T06:19:34.253631+0000","last_clean":"2024-09-16T06:19:34.253631+0000","last_became_active":"2024-09-16T06:19:21.398252+0000","last_became_peered":"2024-09-16T06:19:21.398252+0000","last_unstale":"2024-09-16T06:19:34.253631+0000","last_undegraded":"2024-09-16T06:19:34.253631+0000","last_fullsized":"2024-09-16T06:19:34.253631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:55:33.202540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551948+0000","last_change":"2024-09-16T06:19:25.408462+0000","last_active":"2024-09-16T06:20:39.551948+0000","last_peered":"2024-09-16T06:20:39.551948+0000","last_clean":"2024-09-16T06:20:39.551948+0000","last_became_active":"2024-09-16T06:19:25.407924+0000","last_became_peered":"2024-09-16T06:19:25.407924+0000","last_unstale":"2024-09-16T06:20:39.551948+0000","last_undegraded":"2024-09-16T06:20:39.551948+0000","last_fullsized":"2024-09-16T06:20:39.551948+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:17.213277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542216+0000","last_change":"2024-09-16T06:19:27.424241+0000","last_active":"2024-09-16T06:19:34.542216+0000","last_peered":"2024-09-16T06:19:34.542216+0000","last_clean":"2024-09-16T06:19:34.542216+0000","last_became_active":"2024-09-16T06:19:27.423835+0000","last_became_peered":"2024-09-16T06:19:27.423835+0000","last_unstale":"2024-09-16T06:19:34.542216+0000","last_undegraded":"2024-09-16T06:19:34.542216+0000","last_fullsized":"2024-09-16T06:19:34.542216+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:08.709722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":82,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551907+0000","last_change":"2024-09-16T06:19:23.407553+0000","last_active":"2024-09-16T06:20:39.551907+0000","last_peered":"2024-09-16T06:20:39.551907+0000","last_clean":"2024-09-16T06:20:39.551907+0000","last_became_active":"2024-09-16T06:19:23.407282+0000","last_became_peered":"2024-09-16T06:19:23.407282+0000","last_unstale":"2024-09-16T06:20:39.551907+0000","last_undegraded":"2024-09-16T06:20:39.551907+0000","last_fullsized":"2024-09-16T06:20:39.551907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:15.977324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251586+0000","last_change":"2024-09-16T06:19:21.397155+0000","last_active":"2024-09-16T06:19:34.251586+0000","last_peered":"2024-09-16T06:19:34.251586+0000","last_clean":"2024-09-16T06:19:34.251586+0000","last_became_active":"2024-09-16T06:19:21.396313+0000","last_became_peered":"2024-09-16T06:19:21.396313+0000","last_unstale":"2024-09-16T06:19:34.251586+0000","last_undegraded":"2024-09-16T06:19:34.251586+0000","last_fullsized":"2024-09-16T06:19:34.251586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:43.307627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543234+0000","last_change":"2024-09-16T06:19:25.408185+0000","last_active":"2024-09-16T06:19:34.543234+0000","last_peered":"2024-09-16T06:19:34.543234+0000","last_clean":"2024-09-16T06:19:34.543234+0000","last_became_active":"2024-09-16T06:19:25.407885+0000","last_became_peered":"2024-09-16T06:19:25.407885+0000","last_unstale":"2024-09-16T06:19:34.543234+0000","last_undegraded":"2024-09-16T06:19:34.543234+0000","last_fullsized":"2024-09-16T06:19:34.543234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:01.560872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543711+0000","last_change":"2024-09-16T06:19:27.421212+0000","last_active":"2024-09-16T06:19:34.543711+0000","last_peered":"2024-09-16T06:19:34.543711+0000","last_clean":"2024-09-16T06:19:34.543711+0000","last_became_active":"2024-09-16T06:19:27.420929+0000","last_became_peered":"2024-09-16T06:19:27.420929+0000","last_unstale":"2024-09-16T06:19:34.543711+0000","last_undegraded":"2024-09-16T06:19:34.543711+0000","last_fullsized":"2024-09-16T06:19:34.543711+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:22:59.185946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":44,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254157+0000","last_change":"2024-09-16T06:19:23.399957+0000","last_active":"2024-09-16T06:19:34.254157+0000","last_peered":"2024-09-16T06:19:34.254157+0000","last_clean":"2024-09-16T06:19:34.254157+0000","last_became_active":"2024-09-16T06:19:23.399680+0000","last_became_peered":"2024-09-16T06:19:23.399680+0000","last_unstale":"2024-09-16T06:19:34.254157+0000","last_undegraded":"2024-09-16T06:19:34.254157+0000","last_fullsized":"2024-09-16T06:19:34.254157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:12.684835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":104,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:43.517428+0000","last_change":"2024-09-16T06:19:25.406708+0000","last_active":"2024-09-16T06:20:43.517428+0000","last_peered":"2024-09-16T06:20:43.517428+0000","last_clean":"2024-09-16T06:20:43.517428+0000","last_became_active":"2024-09-16T06:19:21.395420+0000","last_became_peered":"2024-09-16T06:19:21.395420+0000","last_unstale":"2024-09-16T06:20:43.517428+0000","last_undegraded":"2024-09-16T06:20:43.517428+0000","last_fullsized":"2024-09-16T06:20:43.517428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:55.321009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068908200000000004,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253611+0000","last_change":"2024-09-16T06:19:21.399103+0000","last_active":"2024-09-16T06:19:34.253611+0000","last_peered":"2024-09-16T06:19:34.253611+0000","last_clean":"2024-09-16T06:19:34.253611+0000","last_became_active":"2024-09-16T06:19:21.398831+0000","last_became_peered":"2024-09-16T06:19:21.398831+0000","last_unstale":"2024-09-16T06:19:34.253611+0000","last_undegraded":"2024-09-16T06:19:34.253611+0000","last_fullsized":"2024-09-16T06:19:34.253611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:03.819465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254012+0000","last_change":"2024-09-16T06:19:25.410642+0000","last_active":"2024-09-16T06:19:34.254012+0000","last_peered":"2024-09-16T06:19:34.254012+0000","last_clean":"2024-09-16T06:19:34.254012+0000","last_became_active":"2024-09-16T06:19:25.410355+0000","last_became_peered":"2024-09-16T06:19:25.410355+0000","last_unstale":"2024-09-16T06:19:34.254012+0000","last_undegraded":"2024-09-16T06:19:34.254012+0000","last_fullsized":"2024-09-16T06:19:34.254012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:43.062687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542688+0000","last_change":"2024-09-16T06:19:27.778671+0000","last_active":"2024-09-16T06:19:34.542688+0000","last_peered":"2024-09-16T06:19:34.542688+0000","last_clean":"2024-09-16T06:19:34.542688+0000","last_became_active":"2024-09-16T06:19:27.778336+0000","last_became_peered":"2024-09-16T06:19:27.778336+0000","last_unstale":"2024-09-16T06:19:34.542688+0000","last_undegraded":"2024-09-16T06:19:34.542688+0000","last_fullsized":"2024-09-16T06:19:34.542688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:54.249497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":35,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543793+0000","last_change":"2024-09-16T06:19:23.398835+0000","last_active":"2024-09-16T06:19:34.543793+0000","last_peered":"2024-09-16T06:19:34.543793+0000","last_clean":"2024-09-16T06:19:34.543793+0000","last_became_active":"2024-09-16T06:19:23.398511+0000","last_became_peered":"2024-09-16T06:19:23.398511+0000","last_unstale":"2024-09-16T06:19:34.543793+0000","last_undegraded":"2024-09-16T06:19:34.543793+0000","last_fullsized":"2024-09-16T06:19:34.543793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:58:36.714926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":25,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543560+0000","last_change":"2024-09-16T06:19:25.397601+0000","last_active":"2024-09-16T06:19:34.543560+0000","last_peered":"2024-09-16T06:19:34.543560+0000","last_clean":"2024-09-16T06:19:34.543560+0000","last_became_active":"2024-09-16T06:19:21.400521+0000","last_became_peered":"2024-09-16T06:19:21.400521+0000","last_unstale":"2024-09-16T06:19:34.543560+0000","last_undegraded":"2024-09-16T06:19:34.543560+0000","last_fullsized":"2024-09-16T06:19:34.543560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:31:56.227673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048356799999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541987+0000","last_change":"2024-09-16T06:19:21.396761+0000","last_active":"2024-09-16T06:19:34.541987+0000","last_peered":"2024-09-16T06:19:34.541987+0000","last_clean":"2024-09-16T06:19:34.541987+0000","last_became_active":"2024-09-16T06:19:21.396505+0000","last_became_peered":"2024-09-16T06:19:21.396505+0000","last_unstale":"2024-09-16T06:19:34.541987+0000","last_undegraded":"2024-09-16T06:19:34.541987+0000","last_fullsized":"2024-09-16T06:19:34.541987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:22.462518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252002+0000","last_change":"2024-09-16T06:19:25.413602+0000","last_active":"2024-09-16T06:19:34.252002+0000","last_peered":"2024-09-16T06:19:34.252002+0000","last_clean":"2024-09-16T06:19:34.252002+0000","last_became_active":"2024-09-16T06:19:25.413359+0000","last_became_peered":"2024-09-16T06:19:25.413359+0000","last_unstale":"2024-09-16T06:19:34.252002+0000","last_undegraded":"2024-09-16T06:19:34.252002+0000","last_fullsized":"2024-09-16T06:19:34.252002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:50.911744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253234+0000","last_change":"2024-09-16T06:19:27.426230+0000","last_active":"2024-09-16T06:19:34.253234+0000","last_peered":"2024-09-16T06:19:34.253234+0000","last_clean":"2024-09-16T06:19:34.253234+0000","last_became_active":"2024-09-16T06:19:27.425869+0000","last_became_peered":"2024-09-16T06:19:27.425869+0000","last_unstale":"2024-09-16T06:19:34.253234+0000","last_undegraded":"2024-09-16T06:19:34.253234+0000","last_fullsized":"2024-09-16T06:19:34.253234+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:04.155440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'14","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291036+0000","last_change":"2024-09-16T06:19:23.402472+0000","last_active":"2024-09-16T06:19:34.291036+0000","last_peered":"2024-09-16T06:19:34.291036+0000","last_clean":"2024-09-16T06:19:34.291036+0000","last_became_active":"2024-09-16T06:19:23.402013+0000","last_became_peered":"2024-09-16T06:19:23.402013+0000","last_unstale":"2024-09-16T06:19:34.291036+0000","last_undegraded":"2024-09-16T06:19:34.291036+0000","last_fullsized":"2024-09-16T06:19:34.291036+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:10.870975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542899+0000","last_change":"2024-09-16T06:19:25.407357+0000","last_active":"2024-09-16T06:19:34.542899+0000","last_peered":"2024-09-16T06:19:34.542899+0000","last_clean":"2024-09-16T06:19:34.542899+0000","last_became_active":"2024-09-16T06:19:21.399439+0000","last_became_peered":"2024-09-16T06:19:21.399439+0000","last_unstale":"2024-09-16T06:19:34.542899+0000","last_undegraded":"2024-09-16T06:19:34.542899+0000","last_fullsized":"2024-09-16T06:19:34.542899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:52.204830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035565369999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251602+0000","last_change":"2024-09-16T06:19:21.398672+0000","last_active":"2024-09-16T06:19:34.251602+0000","last_peered":"2024-09-16T06:19:34.251602+0000","last_clean":"2024-09-16T06:19:34.251602+0000","last_became_active":"2024-09-16T06:19:21.398171+0000","last_became_peered":"2024-09-16T06:19:21.398171+0000","last_unstale":"2024-09-16T06:19:34.251602+0000","last_undegraded":"2024-09-16T06:19:34.251602+0000","last_fullsized":"2024-09-16T06:19:34.251602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:52.185621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'188","reported_seq":334,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:00.641768+0000","last_change":"2024-09-16T06:19:12.255831+0000","last_active":"2024-09-16T06:20:00.641768+0000","last_peered":"2024-09-16T06:20:00.641768+0000","last_clean":"2024-09-16T06:20:00.641768+0000","last_became_active":"2024-09-16T06:19:12.255346+0000","last_became_peered":"2024-09-16T06:19:12.255346+0000","last_unstale":"2024-09-16T06:20:00.641768+0000","last_undegraded":"2024-09-16T06:20:00.641768+0000","last_fullsized":"2024-09-16T06:20:00.641768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:17:09.239802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:17:09.239802+0000","last_clean_scrub_stamp":"2024-09-16T06:17:09.239802+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:36.608827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542443+0000","last_change":"2024-09-16T06:19:25.413574+0000","last_active":"2024-09-16T06:19:34.542443+0000","last_peered":"2024-09-16T06:19:34.542443+0000","last_clean":"2024-09-16T06:19:34.542443+0000","last_became_active":"2024-09-16T06:19:25.413274+0000","last_became_peered":"2024-09-16T06:19:25.413274+0000","last_unstale":"2024-09-16T06:19:34.542443+0000","last_undegraded":"2024-09-16T06:19:34.542443+0000","last_fullsized":"2024-09-16T06:19:34.542443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:47.539928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.252064+0000","last_change":"2024-09-16T06:19:27.422370+0000","last_active":"2024-09-16T06:19:34.252064+0000","last_peered":"2024-09-16T06:19:34.252064+0000","last_clean":"2024-09-16T06:19:34.252064+0000","last_became_active":"2024-09-16T06:19:27.422088+0000","last_became_peered":"2024-09-16T06:19:27.422088+0000","last_unstale":"2024-09-16T06:19:34.252064+0000","last_undegraded":"2024-09-16T06:19:34.252064+0000","last_fullsized":"2024-09-16T06:19:34.252064+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:26.241616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":48,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290923+0000","last_change":"2024-09-16T06:19:23.401999+0000","last_active":"2024-09-16T06:19:34.290923+0000","last_peered":"2024-09-16T06:19:34.290923+0000","last_clean":"2024-09-16T06:19:34.290923+0000","last_became_active":"2024-09-16T06:19:23.401423+0000","last_became_peered":"2024-09-16T06:19:23.401423+0000","last_unstale":"2024-09-16T06:19:34.290923+0000","last_undegraded":"2024-09-16T06:19:34.290923+0000","last_fullsized":"2024-09-16T06:19:34.290923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:26.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254631+0000","last_change":"2024-09-16T06:19:21.402262+0000","last_active":"2024-09-16T06:19:34.254631+0000","last_peered":"2024-09-16T06:19:34.254631+0000","last_clean":"2024-09-16T06:19:34.254631+0000","last_became_active":"2024-09-16T06:19:21.401980+0000","last_became_peered":"2024-09-16T06:19:21.401980+0000","last_unstale":"2024-09-16T06:19:34.254631+0000","last_undegraded":"2024-09-16T06:19:34.254631+0000","last_fullsized":"2024-09-16T06:19:34.254631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:55.910726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253390+0000","last_change":"2024-09-16T06:19:25.405452+0000","last_active":"2024-09-16T06:19:34.253390+0000","last_peered":"2024-09-16T06:19:34.253390+0000","last_clean":"2024-09-16T06:19:34.253390+0000","last_became_active":"2024-09-16T06:19:25.405172+0000","last_became_peered":"2024-09-16T06:19:25.405172+0000","last_unstale":"2024-09-16T06:19:34.253390+0000","last_undegraded":"2024-09-16T06:19:34.253390+0000","last_fullsized":"2024-09-16T06:19:34.253390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:21.097918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542009+0000","last_change":"2024-09-16T06:19:27.423823+0000","last_active":"2024-09-16T06:19:34.542009+0000","last_peered":"2024-09-16T06:19:34.542009+0000","last_clean":"2024-09-16T06:19:34.542009+0000","last_became_active":"2024-09-16T06:19:27.423507+0000","last_became_peered":"2024-09-16T06:19:27.423507+0000","last_unstale":"2024-09-16T06:19:34.542009+0000","last_undegraded":"2024-09-16T06:19:34.542009+0000","last_fullsized":"2024-09-16T06:19:34.542009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:44.696578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543138+0000","last_change":"2024-09-16T06:19:23.402486+0000","last_active":"2024-09-16T06:19:34.543138+0000","last_peered":"2024-09-16T06:19:34.543138+0000","last_clean":"2024-09-16T06:19:34.543138+0000","last_became_active":"2024-09-16T06:19:23.402166+0000","last_became_peered":"2024-09-16T06:19:23.402166+0000","last_unstale":"2024-09-16T06:19:34.543138+0000","last_undegraded":"2024-09-16T06:19:34.543138+0000","last_fullsized":"2024-09-16T06:19:34.543138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:24.452315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253780+0000","last_change":"2024-09-16T06:19:21.398668+0000","last_active":"2024-09-16T06:19:34.253780+0000","last_peered":"2024-09-16T06:19:34.253780+0000","last_clean":"2024-09-16T06:19:34.253780+0000","last_became_active":"2024-09-16T06:19:21.398193+0000","last_became_peered":"2024-09-16T06:19:21.398193+0000","last_unstale":"2024-09-16T06:19:34.253780+0000","last_undegraded":"2024-09-16T06:19:34.253780+0000","last_fullsized":"2024-09-16T06:19:34.253780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.150203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253937+0000","last_change":"2024-09-16T06:19:25.411973+0000","last_active":"2024-09-16T06:19:34.253937+0000","last_peered":"2024-09-16T06:19:34.253937+0000","last_clean":"2024-09-16T06:19:34.253937+0000","last_became_active":"2024-09-16T06:19:25.411709+0000","last_became_peered":"2024-09-16T06:19:25.411709+0000","last_unstale":"2024-09-16T06:19:34.253937+0000","last_undegraded":"2024-09-16T06:19:34.253937+0000","last_fullsized":"2024-09-16T06:19:34.253937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:22.411633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543401+0000","last_change":"2024-09-16T06:19:27.420324+0000","last_active":"2024-09-16T06:19:34.543401+0000","last_peered":"2024-09-16T06:19:34.543401+0000","last_clean":"2024-09-16T06:19:34.543401+0000","last_became_active":"2024-09-16T06:19:27.420009+0000","last_became_peered":"2024-09-16T06:19:27.420009+0000","last_unstale":"2024-09-16T06:19:34.543401+0000","last_undegraded":"2024-09-16T06:19:34.543401+0000","last_fullsized":"2024-09-16T06:19:34.543401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:00.550562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251279+0000","last_change":"2024-09-16T06:19:23.400938+0000","last_active":"2024-09-16T06:19:34.251279+0000","last_peered":"2024-09-16T06:19:34.251279+0000","last_clean":"2024-09-16T06:19:34.251279+0000","last_became_active":"2024-09-16T06:19:23.400531+0000","last_became_peered":"2024-09-16T06:19:23.400531+0000","last_unstale":"2024-09-16T06:19:34.251279+0000","last_undegraded":"2024-09-16T06:19:34.251279+0000","last_fullsized":"2024-09-16T06:19:34.251279+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:38.033621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254225+0000","last_change":"2024-09-16T06:19:21.398464+0000","last_active":"2024-09-16T06:19:34.254225+0000","last_peered":"2024-09-16T06:19:34.254225+0000","last_clean":"2024-09-16T06:19:34.254225+0000","last_became_active":"2024-09-16T06:19:21.398194+0000","last_became_peered":"2024-09-16T06:19:21.398194+0000","last_unstale":"2024-09-16T06:19:34.254225+0000","last_undegraded":"2024-09-16T06:19:34.254225+0000","last_fullsized":"2024-09-16T06:19:34.254225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:33.051658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551831+0000","last_change":"2024-09-16T06:19:25.411530+0000","last_active":"2024-09-16T06:20:39.551831+0000","last_peered":"2024-09-16T06:20:39.551831+0000","last_clean":"2024-09-16T06:20:39.551831+0000","last_became_active":"2024-09-16T06:19:25.411245+0000","last_became_peered":"2024-09-16T06:19:25.411245+0000","last_unstale":"2024-09-16T06:20:39.551831+0000","last_undegraded":"2024-09-16T06:20:39.551831+0000","last_fullsized":"2024-09-16T06:20:39.551831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:09.237659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251361+0000","last_change":"2024-09-16T06:19:27.779842+0000","last_active":"2024-09-16T06:19:34.251361+0000","last_peered":"2024-09-16T06:19:34.251361+0000","last_clean":"2024-09-16T06:19:34.251361+0000","last_became_active":"2024-09-16T06:19:27.779548+0000","last_became_peered":"2024-09-16T06:19:27.779548+0000","last_unstale":"2024-09-16T06:19:34.251361+0000","last_undegraded":"2024-09-16T06:19:34.251361+0000","last_fullsized":"2024-09-16T06:19:34.251361+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:43.686092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541818+0000","last_change":"2024-09-16T06:19:23.410570+0000","last_active":"2024-09-16T06:19:34.541818+0000","last_peered":"2024-09-16T06:19:34.541818+0000","last_clean":"2024-09-16T06:19:34.541818+0000","last_became_active":"2024-09-16T06:19:23.410253+0000","last_became_peered":"2024-09-16T06:19:23.410253+0000","last_unstale":"2024-09-16T06:19:34.541818+0000","last_undegraded":"2024-09-16T06:19:34.541818+0000","last_fullsized":"2024-09-16T06:19:34.541818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:09.015298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542960+0000","last_change":"2024-09-16T06:19:21.394627+0000","last_active":"2024-09-16T06:19:34.542960+0000","last_peered":"2024-09-16T06:19:34.542960+0000","last_clean":"2024-09-16T06:19:34.542960+0000","last_became_active":"2024-09-16T06:19:21.394367+0000","last_became_peered":"2024-09-16T06:19:21.394367+0000","last_unstale":"2024-09-16T06:19:34.542960+0000","last_undegraded":"2024-09-16T06:19:34.542960+0000","last_fullsized":"2024-09-16T06:19:34.542960+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:31.746257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253337+0000","last_change":"2024-09-16T06:19:25.411366+0000","last_active":"2024-09-16T06:19:34.253337+0000","last_peered":"2024-09-16T06:19:34.253337+0000","last_clean":"2024-09-16T06:19:34.253337+0000","last_became_active":"2024-09-16T06:19:25.411081+0000","last_became_peered":"2024-09-16T06:19:25.411081+0000","last_unstale":"2024-09-16T06:19:34.253337+0000","last_undegraded":"2024-09-16T06:19:34.253337+0000","last_fullsized":"2024-09-16T06:19:34.253337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:50.158742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542607+0000","last_change":"2024-09-16T06:19:27.422993+0000","last_active":"2024-09-16T06:19:34.542607+0000","last_peered":"2024-09-16T06:19:34.542607+0000","last_clean":"2024-09-16T06:19:34.542607+0000","last_became_active":"2024-09-16T06:19:27.421604+0000","last_became_peered":"2024-09-16T06:19:27.421604+0000","last_unstale":"2024-09-16T06:19:34.542607+0000","last_undegraded":"2024-09-16T06:19:34.542607+0000","last_fullsized":"2024-09-16T06:19:34.542607+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:55.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543685+0000","last_change":"2024-09-16T06:19:23.404751+0000","last_active":"2024-09-16T06:19:34.543685+0000","last_peered":"2024-09-16T06:19:34.543685+0000","last_clean":"2024-09-16T06:19:34.543685+0000","last_became_active":"2024-09-16T06:19:23.404205+0000","last_became_peered":"2024-09-16T06:19:23.404205+0000","last_unstale":"2024-09-16T06:19:34.543685+0000","last_undegraded":"2024-09-16T06:19:34.543685+0000","last_fullsized":"2024-09-16T06:19:34.543685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:32.881032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253492+0000","last_change":"2024-09-16T06:19:21.402169+0000","last_active":"2024-09-16T06:19:34.253492+0000","last_peered":"2024-09-16T06:19:34.253492+0000","last_clean":"2024-09-16T06:19:34.253492+0000","last_became_active":"2024-09-16T06:19:21.401883+0000","last_became_peered":"2024-09-16T06:19:21.401883+0000","last_unstale":"2024-09-16T06:19:34.253492+0000","last_undegraded":"2024-09-16T06:19:34.253492+0000","last_fullsized":"2024-09-16T06:19:34.253492+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:32:14.708919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.544025+0000","last_change":"2024-09-16T06:19:25.409659+0000","last_active":"2024-09-16T06:19:34.544025+0000","last_peered":"2024-09-16T06:19:34.544025+0000","last_clean":"2024-09-16T06:19:34.544025+0000","last_became_active":"2024-09-16T06:19:25.409404+0000","last_became_peered":"2024-09-16T06:19:25.409404+0000","last_unstale":"2024-09-16T06:19:34.544025+0000","last_undegraded":"2024-09-16T06:19:34.544025+0000","last_fullsized":"2024-09-16T06:19:34.544025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:05.792174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254139+0000","last_change":"2024-09-16T06:19:27.416443+0000","last_active":"2024-09-16T06:19:34.254139+0000","last_peered":"2024-09-16T06:19:34.254139+0000","last_clean":"2024-09-16T06:19:34.254139+0000","last_became_active":"2024-09-16T06:19:27.416184+0000","last_became_peered":"2024-09-16T06:19:27.416184+0000","last_unstale":"2024-09-16T06:19:34.254139+0000","last_undegraded":"2024-09-16T06:19:34.254139+0000","last_fullsized":"2024-09-16T06:19:34.254139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:46.453311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290840+0000","last_change":"2024-09-16T06:19:21.397729+0000","last_active":"2024-09-16T06:19:34.290840+0000","last_peered":"2024-09-16T06:19:34.290840+0000","last_clean":"2024-09-16T06:19:34.290840+0000","last_became_active":"2024-09-16T06:19:21.397482+0000","last_became_peered":"2024-09-16T06:19:21.397482+0000","last_unstale":"2024-09-16T06:19:34.290840+0000","last_undegraded":"2024-09-16T06:19:34.290840+0000","last_fullsized":"2024-09-16T06:19:34.290840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:48.615614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.544165+0000","last_change":"2024-09-16T06:19:23.408845+0000","last_active":"2024-09-16T06:19:34.544165+0000","last_peered":"2024-09-16T06:19:34.544165+0000","last_clean":"2024-09-16T06:19:34.544165+0000","last_became_active":"2024-09-16T06:19:23.407496+0000","last_became_peered":"2024-09-16T06:19:23.407496+0000","last_unstale":"2024-09-16T06:19:34.544165+0000","last_undegraded":"2024-09-16T06:19:34.544165+0000","last_fullsized":"2024-09-16T06:19:34.544165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:56:41.487842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251933+0000","last_change":"2024-09-16T06:19:25.415353+0000","last_active":"2024-09-16T06:19:34.251933+0000","last_peered":"2024-09-16T06:19:34.251933+0000","last_clean":"2024-09-16T06:19:34.251933+0000","last_became_active":"2024-09-16T06:19:25.415050+0000","last_became_peered":"2024-09-16T06:19:25.415050+0000","last_unstale":"2024-09-16T06:19:34.251933+0000","last_undegraded":"2024-09-16T06:19:34.251933+0000","last_fullsized":"2024-09-16T06:19:34.251933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:19.735099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542619+0000","last_change":"2024-09-16T06:19:27.780194+0000","last_active":"2024-09-16T06:19:34.542619+0000","last_peered":"2024-09-16T06:19:34.542619+0000","last_clean":"2024-09-16T06:19:34.542619+0000","last_became_active":"2024-09-16T06:19:27.779890+0000","last_became_peered":"2024-09-16T06:19:27.779890+0000","last_unstale":"2024-09-16T06:19:34.542619+0000","last_undegraded":"2024-09-16T06:19:34.542619+0000","last_fullsized":"2024-09-16T06:19:34.542619+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:38.244708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543509+0000","last_change":"2024-09-16T06:19:23.411528+0000","last_active":"2024-09-16T06:19:34.543509+0000","last_peered":"2024-09-16T06:19:34.543509+0000","last_clean":"2024-09-16T06:19:34.543509+0000","last_became_active":"2024-09-16T06:19:23.411257+0000","last_became_peered":"2024-09-16T06:19:23.411257+0000","last_unstale":"2024-09-16T06:19:34.543509+0000","last_undegraded":"2024-09-16T06:19:34.543509+0000","last_fullsized":"2024-09-16T06:19:34.543509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:18.438834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251573+0000","last_change":"2024-09-16T06:19:21.398837+0000","last_active":"2024-09-16T06:19:34.251573+0000","last_peered":"2024-09-16T06:19:34.251573+0000","last_clean":"2024-09-16T06:19:34.251573+0000","last_became_active":"2024-09-16T06:19:21.398111+0000","last_became_peered":"2024-09-16T06:19:21.398111+0000","last_unstale":"2024-09-16T06:19:34.251573+0000","last_undegraded":"2024-09-16T06:19:34.251573+0000","last_fullsized":"2024-09-16T06:19:34.251573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:37:07.316318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551837+0000","last_change":"2024-09-16T06:19:25.410242+0000","last_active":"2024-09-16T06:20:39.551837+0000","last_peered":"2024-09-16T06:20:39.551837+0000","last_clean":"2024-09-16T06:20:39.551837+0000","last_became_active":"2024-09-16T06:19:25.409777+0000","last_became_peered":"2024-09-16T06:19:25.409777+0000","last_unstale":"2024-09-16T06:20:39.551837+0000","last_undegraded":"2024-09-16T06:20:39.551837+0000","last_fullsized":"2024-09-16T06:20:39.551837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:15.072946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543370+0000","last_change":"2024-09-16T06:19:27.423715+0000","last_active":"2024-09-16T06:19:34.543370+0000","last_peered":"2024-09-16T06:19:34.543370+0000","last_clean":"2024-09-16T06:19:34.543370+0000","last_became_active":"2024-09-16T06:19:27.423414+0000","last_became_peered":"2024-09-16T06:19:27.423414+0000","last_unstale":"2024-09-16T06:19:34.543370+0000","last_undegraded":"2024-09-16T06:19:34.543370+0000","last_fullsized":"2024-09-16T06:19:34.543370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:42.915520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251891+0000","last_change":"2024-09-16T06:19:23.402973+0000","last_active":"2024-09-16T06:19:34.251891+0000","last_peered":"2024-09-16T06:19:34.251891+0000","last_clean":"2024-09-16T06:19:34.251891+0000","last_became_active":"2024-09-16T06:19:23.402651+0000","last_became_peered":"2024-09-16T06:19:23.402651+0000","last_unstale":"2024-09-16T06:19:34.251891+0000","last_undegraded":"2024-09-16T06:19:34.251891+0000","last_fullsized":"2024-09-16T06:19:34.251891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:28.773138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543271+0000","last_change":"2024-09-16T06:19:21.397191+0000","last_active":"2024-09-16T06:19:34.543271+0000","last_peered":"2024-09-16T06:19:34.543271+0000","last_clean":"2024-09-16T06:19:34.543271+0000","last_became_active":"2024-09-16T06:19:21.396932+0000","last_became_peered":"2024-09-16T06:19:21.396932+0000","last_unstale":"2024-09-16T06:19:34.543271+0000","last_undegraded":"2024-09-16T06:19:34.543271+0000","last_fullsized":"2024-09-16T06:19:34.543271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:19.912733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.552054+0000","last_change":"2024-09-16T06:19:25.408874+0000","last_active":"2024-09-16T06:20:39.552054+0000","last_peered":"2024-09-16T06:20:39.552054+0000","last_clean":"2024-09-16T06:20:39.552054+0000","last_became_active":"2024-09-16T06:19:25.407793+0000","last_became_peered":"2024-09-16T06:19:25.407793+0000","last_unstale":"2024-09-16T06:20:39.552054+0000","last_undegraded":"2024-09-16T06:20:39.552054+0000","last_fullsized":"2024-09-16T06:20:39.552054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:50.017797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542678+0000","last_change":"2024-09-16T06:19:27.416993+0000","last_active":"2024-09-16T06:19:34.542678+0000","last_peered":"2024-09-16T06:19:34.542678+0000","last_clean":"2024-09-16T06:19:34.542678+0000","last_became_active":"2024-09-16T06:19:27.416759+0000","last_became_peered":"2024-09-16T06:19:27.416759+0000","last_unstale":"2024-09-16T06:19:34.542678+0000","last_undegraded":"2024-09-16T06:19:34.542678+0000","last_fullsized":"2024-09-16T06:19:34.542678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:30.611372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254002+0000","last_change":"2024-09-16T06:19:23.401460+0000","last_active":"2024-09-16T06:19:34.254002+0000","last_peered":"2024-09-16T06:19:34.254002+0000","last_clean":"2024-09-16T06:19:34.254002+0000","last_became_active":"2024-09-16T06:19:23.401168+0000","last_became_peered":"2024-09-16T06:19:23.401168+0000","last_unstale":"2024-09-16T06:19:34.254002+0000","last_undegraded":"2024-09-16T06:19:34.254002+0000","last_fullsized":"2024-09-16T06:19:34.254002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:42.903240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291120+0000","last_change":"2024-09-16T06:19:21.405127+0000","last_active":"2024-09-16T06:19:34.291120+0000","last_peered":"2024-09-16T06:19:34.291120+0000","last_clean":"2024-09-16T06:19:34.291120+0000","last_became_active":"2024-09-16T06:19:21.404924+0000","last_became_peered":"2024-09-16T06:19:21.404924+0000","last_unstale":"2024-09-16T06:19:34.291120+0000","last_undegraded":"2024-09-16T06:19:34.291120+0000","last_fullsized":"2024-09-16T06:19:34.291120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:35.406875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542662+0000","last_change":"2024-09-16T06:19:25.410353+0000","last_active":"2024-09-16T06:19:34.542662+0000","last_peered":"2024-09-16T06:19:34.542662+0000","last_clean":"2024-09-16T06:19:34.542662+0000","last_became_active":"2024-09-16T06:19:25.410058+0000","last_became_peered":"2024-09-16T06:19:25.410058+0000","last_unstale":"2024-09-16T06:19:34.542662+0000","last_undegraded":"2024-09-16T06:19:34.542662+0000","last_fullsized":"2024-09-16T06:19:34.542662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:48.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542256+0000","last_change":"2024-09-16T06:19:27.421209+0000","last_active":"2024-09-16T06:19:34.542256+0000","last_peered":"2024-09-16T06:19:34.542256+0000","last_clean":"2024-09-16T06:19:34.542256+0000","last_became_active":"2024-09-16T06:19:27.420885+0000","last_became_peered":"2024-09-16T06:19:27.420885+0000","last_unstale":"2024-09-16T06:19:34.542256+0000","last_undegraded":"2024-09-16T06:19:34.542256+0000","last_fullsized":"2024-09-16T06:19:34.542256+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:09.249936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542247+0000","last_change":"2024-09-16T06:19:23.410647+0000","last_active":"2024-09-16T06:19:34.542247+0000","last_peered":"2024-09-16T06:19:34.542247+0000","last_clean":"2024-09-16T06:19:34.542247+0000","last_became_active":"2024-09-16T06:19:23.410375+0000","last_became_peered":"2024-09-16T06:19:23.410375+0000","last_unstale":"2024-09-16T06:19:34.542247+0000","last_undegraded":"2024-09-16T06:19:34.542247+0000","last_fullsized":"2024-09-16T06:19:34.542247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:59.571144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253682+0000","last_change":"2024-09-16T06:19:21.394709+0000","last_active":"2024-09-16T06:19:34.253682+0000","last_peered":"2024-09-16T06:19:34.253682+0000","last_clean":"2024-09-16T06:19:34.253682+0000","last_became_active":"2024-09-16T06:19:21.393441+0000","last_became_peered":"2024-09-16T06:19:21.393441+0000","last_unstale":"2024-09-16T06:19:34.253682+0000","last_undegraded":"2024-09-16T06:19:34.253682+0000","last_fullsized":"2024-09-16T06:19:34.253682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:19.210905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253835+0000","last_change":"2024-09-16T06:19:25.407913+0000","last_active":"2024-09-16T06:19:34.253835+0000","last_peered":"2024-09-16T06:19:34.253835+0000","last_clean":"2024-09-16T06:19:34.253835+0000","last_became_active":"2024-09-16T06:19:25.407131+0000","last_became_peered":"2024-09-16T06:19:25.407131+0000","last_unstale":"2024-09-16T06:19:34.253835+0000","last_undegraded":"2024-09-16T06:19:34.253835+0000","last_fullsized":"2024-09-16T06:19:34.253835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:42.857972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542269+0000","last_change":"2024-09-16T06:19:27.423373+0000","last_active":"2024-09-16T06:19:34.542269+0000","last_peered":"2024-09-16T06:19:34.542269+0000","last_clean":"2024-09-16T06:19:34.542269+0000","last_became_active":"2024-09-16T06:19:27.423101+0000","last_became_peered":"2024-09-16T06:19:27.423101+0000","last_unstale":"2024-09-16T06:19:34.542269+0000","last_undegraded":"2024-09-16T06:19:34.542269+0000","last_fullsized":"2024-09-16T06:19:34.542269+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:46.996243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542364+0000","last_change":"2024-09-16T06:19:23.407023+0000","last_active":"2024-09-16T06:19:34.542364+0000","last_peered":"2024-09-16T06:19:34.542364+0000","last_clean":"2024-09-16T06:19:34.542364+0000","last_became_active":"2024-09-16T06:19:23.406745+0000","last_became_peered":"2024-09-16T06:19:23.406745+0000","last_unstale":"2024-09-16T06:19:34.542364+0000","last_undegraded":"2024-09-16T06:19:34.542364+0000","last_fullsized":"2024-09-16T06:19:34.542364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.191754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251988+0000","last_change":"2024-09-16T06:19:21.401070+0000","last_active":"2024-09-16T06:19:34.251988+0000","last_peered":"2024-09-16T06:19:34.251988+0000","last_clean":"2024-09-16T06:19:34.251988+0000","last_became_active":"2024-09-16T06:19:21.400799+0000","last_became_peered":"2024-09-16T06:19:21.400799+0000","last_unstale":"2024-09-16T06:19:34.251988+0000","last_undegraded":"2024-09-16T06:19:34.251988+0000","last_fullsized":"2024-09-16T06:19:34.251988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:49.197425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.290649+0000","last_change":"2024-09-16T06:19:25.408421+0000","last_active":"2024-09-16T06:19:34.290649+0000","last_peered":"2024-09-16T06:19:34.290649+0000","last_clean":"2024-09-16T06:19:34.290649+0000","last_became_active":"2024-09-16T06:19:25.408097+0000","last_became_peered":"2024-09-16T06:19:25.408097+0000","last_unstale":"2024-09-16T06:19:34.290649+0000","last_undegraded":"2024-09-16T06:19:34.290649+0000","last_fullsized":"2024-09-16T06:19:34.290649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:37:08.841946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541783+0000","last_change":"2024-09-16T06:19:27.415564+0000","last_active":"2024-09-16T06:19:34.541783+0000","last_peered":"2024-09-16T06:19:34.541783+0000","last_clean":"2024-09-16T06:19:34.541783+0000","last_became_active":"2024-09-16T06:19:27.415274+0000","last_became_peered":"2024-09-16T06:19:27.415274+0000","last_unstale":"2024-09-16T06:19:34.541783+0000","last_undegraded":"2024-09-16T06:19:34.541783+0000","last_fullsized":"2024-09-16T06:19:34.541783+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:37:35.895555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542896+0000","last_change":"2024-09-16T06:19:23.408366+0000","last_active":"2024-09-16T06:19:34.542896+0000","last_peered":"2024-09-16T06:19:34.542896+0000","last_clean":"2024-09-16T06:19:34.542896+0000","last_became_active":"2024-09-16T06:19:23.408005+0000","last_became_peered":"2024-09-16T06:19:23.408005+0000","last_unstale":"2024-09-16T06:19:34.542896+0000","last_undegraded":"2024-09-16T06:19:34.542896+0000","last_fullsized":"2024-09-16T06:19:34.542896+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:40.041095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291190+0000","last_change":"2024-09-16T06:19:21.400031+0000","last_active":"2024-09-16T06:19:34.291190+0000","last_peered":"2024-09-16T06:19:34.291190+0000","last_clean":"2024-09-16T06:19:34.291190+0000","last_became_active":"2024-09-16T06:19:21.398723+0000","last_became_peered":"2024-09-16T06:19:21.398723+0000","last_unstale":"2024-09-16T06:19:34.291190+0000","last_undegraded":"2024-09-16T06:19:34.291190+0000","last_fullsized":"2024-09-16T06:19:34.291190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:34.838879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.541669+0000","last_change":"2024-09-16T06:19:25.409291+0000","last_active":"2024-09-16T06:19:34.541669+0000","last_peered":"2024-09-16T06:19:34.541669+0000","last_clean":"2024-09-16T06:19:34.541669+0000","last_became_active":"2024-09-16T06:19:25.408978+0000","last_became_peered":"2024-09-16T06:19:25.408978+0000","last_unstale":"2024-09-16T06:19:34.541669+0000","last_undegraded":"2024-09-16T06:19:34.541669+0000","last_fullsized":"2024-09-16T06:19:34.541669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:09.397670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542568+0000","last_change":"2024-09-16T06:19:27.780703+0000","last_active":"2024-09-16T06:19:34.542568+0000","last_peered":"2024-09-16T06:19:34.542568+0000","last_clean":"2024-09-16T06:19:34.542568+0000","last_became_active":"2024-09-16T06:19:27.780368+0000","last_became_peered":"2024-09-16T06:19:27.780368+0000","last_unstale":"2024-09-16T06:19:34.542568+0000","last_undegraded":"2024-09-16T06:19:34.542568+0000","last_fullsized":"2024-09-16T06:19:34.542568+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:48.322321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542465+0000","last_change":"2024-09-16T06:19:27.418763+0000","last_active":"2024-09-16T06:19:34.542465+0000","last_peered":"2024-09-16T06:19:34.542465+0000","last_clean":"2024-09-16T06:19:34.542465+0000","last_became_active":"2024-09-16T06:19:27.418473+0000","last_became_peered":"2024-09-16T06:19:27.418473+0000","last_unstale":"2024-09-16T06:19:34.542465+0000","last_undegraded":"2024-09-16T06:19:34.542465+0000","last_fullsized":"2024-09-16T06:19:34.542465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:44.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.251299+0000","last_change":"2024-09-16T06:19:21.395909+0000","last_active":"2024-09-16T06:19:34.251299+0000","last_peered":"2024-09-16T06:19:34.251299+0000","last_clean":"2024-09-16T06:19:34.251299+0000","last_became_active":"2024-09-16T06:19:21.395653+0000","last_became_peered":"2024-09-16T06:19:21.395653+0000","last_unstale":"2024-09-16T06:19:34.251299+0000","last_undegraded":"2024-09-16T06:19:34.251299+0000","last_fullsized":"2024-09-16T06:19:34.251299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:30.813769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543579+0000","last_change":"2024-09-16T06:19:23.405752+0000","last_active":"2024-09-16T06:19:34.543579+0000","last_peered":"2024-09-16T06:19:34.543579+0000","last_clean":"2024-09-16T06:19:34.543579+0000","last_became_active":"2024-09-16T06:19:23.405364+0000","last_became_peered":"2024-09-16T06:19:23.405364+0000","last_unstale":"2024-09-16T06:19:34.543579+0000","last_undegraded":"2024-09-16T06:19:34.543579+0000","last_fullsized":"2024-09-16T06:19:34.543579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:16.689204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T06:20:39.551827+0000","last_change":"2024-09-16T06:19:25.407896+0000","last_active":"2024-09-16T06:20:39.551827+0000","last_peered":"2024-09-16T06:20:39.551827+0000","last_clean":"2024-09-16T06:20:39.551827+0000","last_became_active":"2024-09-16T06:19:25.407591+0000","last_became_peered":"2024-09-16T06:19:25.407591+0000","last_unstale":"2024-09-16T06:20:39.551827+0000","last_undegraded":"2024-09-16T06:20:39.551827+0000","last_fullsized":"2024-09-16T06:20:39.551827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:28.606239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253183+0000","last_change":"2024-09-16T06:19:27.418777+0000","last_active":"2024-09-16T06:19:34.253183+0000","last_peered":"2024-09-16T06:19:34.253183+0000","last_clean":"2024-09-16T06:19:34.253183+0000","last_became_active":"2024-09-16T06:19:27.418444+0000","last_became_peered":"2024-09-16T06:19:27.418444+0000","last_unstale":"2024-09-16T06:19:34.253183+0000","last_undegraded":"2024-09-16T06:19:34.253183+0000","last_fullsized":"2024-09-16T06:19:34.253183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:43.452190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543075+0000","last_change":"2024-09-16T06:19:21.405315+0000","last_active":"2024-09-16T06:19:34.543075+0000","last_peered":"2024-09-16T06:19:34.543075+0000","last_clean":"2024-09-16T06:19:34.543075+0000","last_became_active":"2024-09-16T06:19:21.405046+0000","last_became_peered":"2024-09-16T06:19:21.405046+0000","last_unstale":"2024-09-16T06:19:34.543075+0000","last_undegraded":"2024-09-16T06:19:34.543075+0000","last_fullsized":"2024-09-16T06:19:34.543075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.933552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291091+0000","last_change":"2024-09-16T06:19:23.401764+0000","last_active":"2024-09-16T06:19:34.291091+0000","last_peered":"2024-09-16T06:19:34.291091+0000","last_clean":"2024-09-16T06:19:34.291091+0000","last_became_active":"2024-09-16T06:19:23.401366+0000","last_became_peered":"2024-09-16T06:19:23.401366+0000","last_unstale":"2024-09-16T06:19:34.291091+0000","last_undegraded":"2024-09-16T06:19:34.291091+0000","last_fullsized":"2024-09-16T06:19:34.291091+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:40.312533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543499+0000","last_change":"2024-09-16T06:19:25.408286+0000","last_active":"2024-09-16T06:19:34.543499+0000","last_peered":"2024-09-16T06:19:34.543499+0000","last_clean":"2024-09-16T06:19:34.543499+0000","last_became_active":"2024-09-16T06:19:25.407981+0000","last_became_peered":"2024-09-16T06:19:25.407981+0000","last_unstale":"2024-09-16T06:19:34.543499+0000","last_undegraded":"2024-09-16T06:19:34.543499+0000","last_fullsized":"2024-09-16T06:19:34.543499+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:58.208151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542760+0000","last_change":"2024-09-16T06:19:27.779028+0000","last_active":"2024-09-16T06:19:34.542760+0000","last_peered":"2024-09-16T06:19:34.542760+0000","last_clean":"2024-09-16T06:19:34.542760+0000","last_became_active":"2024-09-16T06:19:27.778712+0000","last_became_peered":"2024-09-16T06:19:27.778712+0000","last_unstale":"2024-09-16T06:19:34.542760+0000","last_undegraded":"2024-09-16T06:19:34.542760+0000","last_fullsized":"2024-09-16T06:19:34.542760+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:49:52.365369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.291152+0000","last_change":"2024-09-16T06:19:21.397572+0000","last_active":"2024-09-16T06:19:34.291152+0000","last_peered":"2024-09-16T06:19:34.291152+0000","last_clean":"2024-09-16T06:19:34.291152+0000","last_became_active":"2024-09-16T06:19:21.397293+0000","last_became_peered":"2024-09-16T06:19:21.397293+0000","last_unstale":"2024-09-16T06:19:34.291152+0000","last_undegraded":"2024-09-16T06:19:34.291152+0000","last_fullsized":"2024-09-16T06:19:34.291152+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:24.952651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.254429+0000","last_change":"2024-09-16T06:19:23.403386+0000","last_active":"2024-09-16T06:19:34.254429+0000","last_peered":"2024-09-16T06:19:34.254429+0000","last_clean":"2024-09-16T06:19:34.254429+0000","last_became_active":"2024-09-16T06:19:23.403081+0000","last_became_peered":"2024-09-16T06:19:23.403081+0000","last_unstale":"2024-09-16T06:19:34.254429+0000","last_undegraded":"2024-09-16T06:19:34.254429+0000","last_fullsized":"2024-09-16T06:19:34.254429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:18.401286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.253562+0000","last_change":"2024-09-16T06:19:25.412344+0000","last_active":"2024-09-16T06:19:34.253562+0000","last_peered":"2024-09-16T06:19:34.253562+0000","last_clean":"2024-09-16T06:19:34.253562+0000","last_became_active":"2024-09-16T06:19:25.412083+0000","last_became_peered":"2024-09-16T06:19:25.412083+0000","last_unstale":"2024-09-16T06:19:34.253562+0000","last_undegraded":"2024-09-16T06:19:34.253562+0000","last_fullsized":"2024-09-16T06:19:34.253562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.639770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543947+0000","last_change":"2024-09-16T06:19:27.778902+0000","last_active":"2024-09-16T06:19:34.543947+0000","last_peered":"2024-09-16T06:19:34.543947+0000","last_clean":"2024-09-16T06:19:34.543947+0000","last_became_active":"2024-09-16T06:19:27.778632+0000","last_became_peered":"2024-09-16T06:19:27.778632+0000","last_unstale":"2024-09-16T06:19:34.543947+0000","last_undegraded":"2024-09-16T06:19:34.543947+0000","last_fullsized":"2024-09-16T06:19:34.543947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:26.394832+0000","last_clean_scrub_stamp":"2024-09-16T06:19:26.394832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:08.990227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543252+0000","last_change":"2024-09-16T06:19:21.398734+0000","last_active":"2024-09-16T06:19:34.543252+0000","last_peered":"2024-09-16T06:19:34.543252+0000","last_clean":"2024-09-16T06:19:34.543252+0000","last_became_active":"2024-09-16T06:19:21.398407+0000","last_became_peered":"2024-09-16T06:19:21.398407+0000","last_unstale":"2024-09-16T06:19:34.543252+0000","last_undegraded":"2024-09-16T06:19:34.543252+0000","last_fullsized":"2024-09-16T06:19:34.543252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:20.374051+0000","last_clean_scrub_stamp":"2024-09-16T06:19:20.374051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:59:48.189326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.543657+0000","last_change":"2024-09-16T06:19:23.408230+0000","last_active":"2024-09-16T06:19:34.543657+0000","last_peered":"2024-09-16T06:19:34.543657+0000","last_clean":"2024-09-16T06:19:34.543657+0000","last_became_active":"2024-09-16T06:19:23.407555+0000","last_became_peered":"2024-09-16T06:19:23.407555+0000","last_unstale":"2024-09-16T06:19:34.543657+0000","last_undegraded":"2024-09-16T06:19:34.543657+0000","last_fullsized":"2024-09-16T06:19:34.543657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:22.382849+0000","last_clean_scrub_stamp":"2024-09-16T06:19:22.382849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:01:06.246213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:19:34.542263+0000","last_change":"2024-09-16T06:19:25.406475+0000","last_active":"2024-09-16T06:19:34.542263+0000","last_peered":"2024-09-16T06:19:34.542263+0000","last_clean":"2024-09-16T06:19:34.542263+0000","last_became_active":"2024-09-16T06:19:25.404650+0000","last_became_peered":"2024-09-16T06:19:25.404650+0000","last_unstale":"2024-09-16T06:19:34.542263+0000","last_undegraded":"2024-09-16T06:19:34.542263+0000","last_fullsized":"2024-09-16T06:19:34.542263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:19:24.388548+0000","last_clean_scrub_stamp":"2024-09-16T06:19:24.388548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:31.426984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201301,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28356,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712700,"statfs":{"total":95990841344,"available":95961804800,"internally_reserved":0,"allocated":1372160,"data_stored":911333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58499999999999996}]},{"osd":1,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.52100000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41499999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55800000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32428,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708628,"statfs":{"total":95990841344,"available":95957635072,"internally_reserved":0,"allocated":1343488,"data_stored":909220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.45700000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.311}]},{"osd":7,"last update":"Mon Sep 16 06:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.61099999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713296,"statfs":{"total":95990841344,"available":95962415104,"internally_reserved":0,"allocated":757760,"data_stored":319384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.34000000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59699999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.498},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48499999999999999},"min":{"1min":0.36699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.46800000000000003,"15min":0.46100000000000002},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63600000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.40600000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.38300000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49299999999999999},"min":{"1min":0.39300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.36799999999999999,"15min":0.36499999999999999},"min":{"1min":0.27000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.32700000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.434}]},{"osd":6,"last update":"Mon Sep 16 06:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713268,"statfs":{"total":95990841344,"available":95962386432,"internally_reserved":0,"allocated":786432,"data_stored":320740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.752,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.502}]},{"osd":1,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Sep 16 06:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.49099999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41299999999999998},"min":{"1min":0.29199999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29999999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59999999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40899999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39100000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.373},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47499999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31892,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709164,"statfs":{"total":95990841344,"available":95958183936,"internally_reserved":0,"allocated":794624,"data_stored":319944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.38200000000000001,"15min":0.40500000000000003},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.49299999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.40100000000000002},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.50900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.29799999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47499999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.25900000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:20:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.434,"15min":0.44800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.53700000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.375},{"interface":"front","average":{"1min":0.37,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.46600000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.311}]},{"osd":4,"last update":"Mon Sep 16 06:19:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.443},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.749,"5min":0.755,"15min":0.755},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.622,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.496}]},{"osd":6,"last update":"Mon Sep 16 06:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713292,"statfs":{"total":95990841344,"available":95962411008,"internally_reserved":0,"allocated":761856,"data_stored":319856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.25}]},{"osd":1,"last update":"Mon Sep 16 06:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.42499999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35899999999999999,"15min":0.35599999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.46999999999999997}]},{"osd":3,"last update":"Mon Sep 16 06:20:35 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40500000000000003,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.39500000000000002,"15min":0.40899999999999997},"min":{"1min":0.25900000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:19:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68000000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.53400000000000003}]},{"osd":5,"last update":"Mon Sep 16 06:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66900000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.438}]},{"osd":6,"last update":"Mon Sep 16 06:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45600000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.621},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.42199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713240,"statfs":{"total":95990841344,"available":95962357760,"internally_reserved":0,"allocated":815104,"data_stored":321434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.318},"min":{"1min":0.23499999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.32100000000000001},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.38500000000000001,"15min":0.38100000000000001},"min":{"1min":0.27800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Sep 16 06:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.27200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Sep 16 06:20:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.5}]},{"osd":5,"last update":"Mon Sep 16 06:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61099999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:19:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.623}]},{"osd":7,"last update":"Mon Sep 16 06:20:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28360,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712696,"statfs":{"total":95990841344,"available":95961800704,"internally_reserved":0,"allocated":1376256,"data_stored":911361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.30399999999999999},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.246},{"interface":"front","average":{"1min":0.373,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52100000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36599999999999999,"15min":0.36199999999999999},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.379,"15min":0.371},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.439},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.36299999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:19:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80400000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.71499999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.439},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40200000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:20:48.230 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T06:20:48.230 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T06:20:48.230 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T06:20:48.231 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T06:20:48.425 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:48.804 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:48 smithi190 ceph-mon[25888]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:20:48.805 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:48 smithi190 ceph-mon[39731]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:20:48.957 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-mon[30578]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T06:20:48.957 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 podman[71626]: 2024-09-16 06:20:48.624111856 +0000 UTC m=+2.281450825 container init 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a, io.openshift.expose-services=, distribution-scope=public, com.redhat.component=ubi8-container, description=Ceph Grafana Container, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, version=8.5, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , build-date=2022-04-29T11:48:11.055897, name=ubi8) 2024-09-16T06:20:48.957 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 podman[71626]: 2024-09-16 06:20:48.630097455 +0000 UTC m=+2.287436412 container start 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a, release=239.1651231664, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, distribution-scope=public, vcs-type=git, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, io.openshift.tags=base rhel8) 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.783829123Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784037043Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784057348Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784064711Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784070715Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784076715Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784082915Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-16T06:20:48.958 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784089245Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784095134Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784101247Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784106696Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784112147Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784117946Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=settings t=2024-09-16T06:20:48.784123897Z level=info msg="App mode production" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=sqlstore t=2024-09-16T06:20:48.784168734Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=sqlstore t=2024-09-16T06:20:48.784184536Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.785019126Z level=info msg="Starting DB migrations" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.785090124Z level=info msg="Executing migration" id="create migration_log table" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.785737171Z level=info msg="Executing migration" id="create user table" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.78620112Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.787045107Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-16T06:20:48.959 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.787896946Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-16T06:20:48.960 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.788698994Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-16T06:20:48.960 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.78966606Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-16T06:20:48.960 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.790589161Z level=info msg="Executing migration" id="create user table v2" 2024-09-16T06:20:48.960 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.791395331Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.792006156Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.792489759Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.793260088Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.793635371Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.794170809Z level=info msg="Executing migration" id="Update user table charset" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.795130323Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.796036812Z level=info msg="Executing migration" id="Add missing user data" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.796756028Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.797738698Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.798460146Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.79946978Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-16T06:20:48.961 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.800772967Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.801256533Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.802062568Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.802930008Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.803374799Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.804198842Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.805034448Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.805909151Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.806791888Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.807627814Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.810587181Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.813251615Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-16T06:20:48.962 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.816211001Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.819002922Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.821626368Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.824550684Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.827169255Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.829982057Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.83032112Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.833206889Z level=info msg="Executing migration" id="create star table" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.833603529Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.835878677Z level=info msg="Executing migration" id="create org table v1" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.838717406Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.839186004Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.839586803Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-16T06:20:48.963 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.840005063Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.840434453Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.84082922Z level=info msg="Executing migration" id="Update org table charset" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.841120171Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.841402515Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.84167885Z level=info msg="Executing migration" id="create dashboard table" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.842028684Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.842452774Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.842899278Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.843336357Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.843797109Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.846427774Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.849204723Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-16T06:20:48.964 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.851763402Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.852204201Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.855021944Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.857271726Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.857699803Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.857971873Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.858456125Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.95785108Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.958681435Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-16T06:20:48.965 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.959290017Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.962018214Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.962473059Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.965110678Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.965420473Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.965725422Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.96799276Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.968795925Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-16T06:20:49.210 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.969554752Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.970390063Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.971234285Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.972029352Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.972535193Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.973357272Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.973798461Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.974220357Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.975151361Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.975587177Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-16T06:20:49.211 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.976432435Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-16T06:20:49.212 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.977293538Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-16T06:20:49.212 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.978039809Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-16T06:20:49.212 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.978424802Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-16T06:20:49.212 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.979216085Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-16T06:20:49.212 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.979688333Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.98001244Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.980372643Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.980895233Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.98145532Z level=info msg="Executing migration" id="create data_source table" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.982305067Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.983107315Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.983589735Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.984001673Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-16T06:20:49.213 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.984432619Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.985477678Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.986408775Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.987151173Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.98758817Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.988468937Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.989223534Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.989755565Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.990761645Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.991180483Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-16T06:20:49.214 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.992088326Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.994946074Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.997645315Z level=info msg="Executing migration" id="Add uid column" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:48 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:48.998197575Z level=info msg="Executing migration" id="Update uid value" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.001029519Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.003378512Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.006263232Z level=info msg="Executing migration" id="create api_key table" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.006694218Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.009468862Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.01237352Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.015153936Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.015561469Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.018293782Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-16T06:20:49.215 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.018729755Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.021239594Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.024080044Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.027034176Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.029603074Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.032383852Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.03272084Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.035099828Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.03793835Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.040451581Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.043120833Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.045775692Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.048387478Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-16T06:20:49.216 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.051318958Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.053892327Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.056728084Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.059406337Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.062109932Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.064958415Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.067294125Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.070020478Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.072833871Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.075337044Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.078075234Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.078865228Z level=info msg="Executing migration" id="create quota table v1" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.07966602Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-16T06:20:49.217 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.080608755Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.081299507Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.081742485Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.082656284Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.083484787Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.084296538Z level=info msg="Executing migration" id="create session table" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.084803991Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.085561107Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.085907649Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.086364369Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.087170901Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.088071889Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.088398689Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.088677511Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.088966335Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-16T06:20:49.218 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.089382941Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.089680104Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.090365619Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.091114071Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.091729273Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.092745369Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.093335278Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.093780828Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.094579621Z level=info msg="Executing migration" id="create alert table v1" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.095523476Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.09617043Z level=info msg="Executing migration" id="add index alert state" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.096975489Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.097425241Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-16T06:20:49.219 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.097817173Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-16T06:20:49.220 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.098317683Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-16T06:20:49.220 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.09876777Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-16T06:20:49.220 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.100301394Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-16T06:20:49.220 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.100696063Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-16T06:20:49.220 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.101173202Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-16T06:20:49.220 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.101506147Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.101860192Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.102260533Z level=info msg="Executing migration" id="Add column is_default" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.104694844Z level=info msg="Executing migration" id="Add column frequency" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.105402666Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.107459126Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.110156211Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.112986354Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.115657433Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-16T06:20:49.221 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.118317563Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.120879339Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.123675352Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.126286381Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.128963007Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.131590337Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.13418757Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.136780743Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.139565001Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.142250749Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.144808235Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.147440849Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-16T06:20:49.222 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.150185044Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.152573672Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.153397647Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.154218403Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.155080046Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.155937287Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.156742912Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.157598932Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.158491381Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.15922466Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.160044263Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.160840808Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.161770626Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.162542025Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-16T06:20:49.223 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.163420519Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.164188534Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.165074056Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.166922605Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.167843294Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.168601325Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.16954954Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.170041546Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.170403067Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.171287476Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.172115003Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.172995267Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.173961212Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-16T06:20:49.224 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.174806058Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.175692657Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.176608489Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.177335084Z level=info msg="Executing migration" id="Move region to single row" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.178285091Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.17905949Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.180000737Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.180714Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.18154373Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.182406114Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.185419452Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.188167091Z level=info msg="Executing migration" id="create test_data table" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.190955153Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-16T06:20:49.225 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.193840347Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.196584979Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.199287938Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.202008815Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.204753076Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.207668927Z level=info msg="Executing migration" id="create team table" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.210129872Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-16T06:20:49.226 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.212904091Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-16T06:20:49.462 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.2157379Z level=info msg="Executing migration" id="create team member table" 2024-09-16T06:20:49.462 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.21870273Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-16T06:20:49.462 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.221321006Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-16T06:20:49.462 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.223996912Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-16T06:20:49.462 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.226924311Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.229718199Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.23245358Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.235009718Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.237720967Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.24059731Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.243144229Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.24574093Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.248222619Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.250887096Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.253423873Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.254056024Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.256763377Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-16T06:20:49.463 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.259313864Z level=info msg="Executing migration" id="create tag table" 2024-09-16T06:20:49.464 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.262114476Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.2626013Z level=info msg="Executing migration" id="create login attempt table" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.265104348Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.267807944Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.270408773Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.27239538Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.273269501Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.274281738Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.274805744Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.275652146Z level=info msg="Executing migration" id="create user auth table" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.276438071Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-16T06:20:49.465 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.27734123Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.278055486Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.27938569Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.28035386Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.281124501Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.281956028Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.282400536Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.283222964Z level=info msg="Executing migration" id="create server_lock table" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.284065618Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.284864381Z level=info msg="Executing migration" id="create user auth token table" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.285343901Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-16T06:20:49.466 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.286189813Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.287049018Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.287928372Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.288700857Z level=info msg="Executing migration" id="create cache_data table" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.289672347Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.290422924Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.291309497Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.291770215Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.292725639Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.293467836Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.294314526Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.294793422Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.295611073Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-16T06:20:49.467 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.296521602Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.297298793Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.297734355Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.298608539Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.299462673Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.300306876Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.300746043Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.303599225Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.306313676Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.309047766Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.309462633Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.309721314Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-16T06:20:49.468 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.310164175Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.310562788Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.310946121Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.311362733Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.312135778Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.312531442Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.312902493Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.316268453Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.319602116Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.320021166Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.320431469Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.321252356Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.321656155Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-16T06:20:49.469 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.32207487Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.322485256Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.322888168Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.323182947Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.323971691Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.324775601Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.325587174Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.325997842Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.326420499Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.327248334Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.328090786Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.328539268Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.329394577Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-16T06:20:49.470 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.330280426Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.330636457Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.331152498Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.331599026Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.332018411Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.332309354Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.334084837Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.335080799Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.335960406Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.336800192Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.337668067Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.337977562Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.338366144Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-16T06:20:49.471 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.339218867Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.339538299Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.340431678Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.340905372Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.343707114Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.346284606Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.349030998Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.349889294Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.352787835Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.355442227Z level=info msg="Executing migration" id="create alert_image table" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.358352772Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.358805525Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.361621663Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.362288038Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-16T06:20:49.472 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.365221231Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.36532674Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.365628579Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.3684236Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.368893748Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.369027496Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.369148685Z level=info msg="alerts found to migrate" alerts=0 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.371416726Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.374128854Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-16T06:20:49.473 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.374631973Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-16T06:20:49.474 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.377155785Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-16T06:20:49.474 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.379756895Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-16T06:20:49.474 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.382399749Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-16T06:20:49.474 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.382716641Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-16T06:20:49.474 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.385544727Z level=info msg="Executing migration" id="create data_keys table" 2024-09-16T06:20:49.474 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.388290914Z level=info msg="Executing migration" id="create secrets table" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.390874608Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.395129016Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.397853763Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.400622205Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.405229969Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.409643164Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.412418016Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.415117376Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.417730424Z level=info msg="Executing migration" id="create permission table" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.418215779Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.419046016Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.419891247Z level=info msg="Executing migration" id="create role table" 2024-09-16T06:20:49.475 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.42069867Z level=info msg="Executing migration" id="add column display_name" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.42166189Z level=info msg="Executing migration" id="add column group_name" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.422608966Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.423456637Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.424431535Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.424992323Z level=info msg="Executing migration" id="create team role table" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.425832826Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.426650636Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.427576635Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.42825941Z level=info msg="Executing migration" id="create user role table" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.429106463Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.429884696Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.430673009Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-16T06:20:49.476 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.431280309Z level=info msg="Executing migration" id="create builtin role table" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.432160993Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.432948244Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.433735301Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.434718941Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.435225122Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.435961156Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.43643111Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.43685696Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.437301031Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.437779653Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.438795166Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.439275097Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-16T06:20:49.477 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.439753672Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.440077651Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.440542771Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.440991012Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.441462128Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.441757127Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.442135713Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.442551534Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.442997995Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.444002484Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.44430858Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.444722624Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.445177652Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-16T06:20:49.478 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.44561637Z level=info msg="Executing migration" id="add correlation config column" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.446590945Z level=info msg="Executing migration" id="create entity_events table" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.447016892Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.447812883Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.447900804Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.448211679Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.448317117Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.448608197Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.448991256Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.449422987Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.449879764Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.45034243Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.450748072Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T06:20:49.479 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.451217067Z level=info msg="Executing migration" id="Drop public config table" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.451650371Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.452104016Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.452578075Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.453065791Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.453895804Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.456936467Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.458129455Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.459147809Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.462048555Z level=info msg="Executing migration" id="add share column" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.464564267Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.467283078Z level=info msg="Executing migration" id="create file table" 2024-09-16T06:20:49.480 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.46787305Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-16T06:20:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-mon[30578]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:49.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.470651082Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-16T06:20:49.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.473434654Z level=info msg="Executing migration" id="create file_meta table" 2024-09-16T06:20:49.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.476169899Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-16T06:20:49.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.476632943Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-16T06:20:49.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.479313446Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-16T06:20:49.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.479725509Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.482266269Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.482939809Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.484962043Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.4852792Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.488208457Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.49084809Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.49354345Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.493992871Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.496784662Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.499494801Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.502156168Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-16T06:20:49.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.512061795Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.514890272Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.517514556Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.521032925Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.521351503Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.524071936Z level=info msg="Executing migration" id="create folder table" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.524490442Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.526852838Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.529458064Z level=info msg="Executing migration" id="Update folder title length" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.529889513Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=migrator t=2024-09-16T06:20:49.532621632Z level=info msg="migrations completed" performed=485 skipped=0 duration=747.54848ms 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=sqlstore t=2024-09-16T06:20:49.533017131Z level=info msg="Created default organization" 2024-09-16T06:20:49.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 bash[71626]: 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 systemd[1]: Started Ceph grafana.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=plugin.loader t=2024-09-16T06:20:49.592082569Z level=info msg="Plugin registered" pluginID=input 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=plugin.loader t=2024-09-16T06:20:49.621833702Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=plugin.loader t=2024-09-16T06:20:49.621869403Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=secrets t=2024-09-16T06:20:49.621934633Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=query_data t=2024-09-16T06:20:49.623723838Z level=info msg="Query Service initialization" 2024-09-16T06:20:49.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:49 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=live.push_http t=2024-09-16T06:20:49.625987797Z level=info msg="Live Push Gateway initialization" 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[25888]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[39731]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:49.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:49 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:50.303 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-16T06:20:50.303 INFO:teuthology.orchestra.run.smithi190.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T06:20:50.664 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=infra.usagestats.collector t=2024-09-16T06:20:50.342409683Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=provisioning.datasources t=2024-09-16T06:20:50.395632675Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=provisioning.datasources t=2024-09-16T06:20:50.428038501Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=provisioning.alerting t=2024-09-16T06:20:50.449369019Z level=info msg="starting to provision alerting" 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=provisioning.alerting t=2024-09-16T06:20:50.449394015Z level=info msg="finished to provision alerting" 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=ngalert.state.manager t=2024-09-16T06:20:50.449503771Z level=info msg="Warming state cache for startup" 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=grafanaStorageLogger t=2024-09-16T06:20:50.449512017Z level=info msg="storage starting" 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=http.server t=2024-09-16T06:20:50.45365907Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=ngalert.state.manager t=2024-09-16T06:20:50.459854054Z level=info msg="State cache has been initialized" states=0 duration=10.348329ms 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=ticker t=2024-09-16T06:20:50.459922282Z level=info msg=starting first_tick=2024-09-16T06:21:00Z 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=ngalert.multiorg.alertmanager t=2024-09-16T06:20:50.459925427Z level=info msg="starting MultiOrg Alertmanager" 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=sqlstore.transactions t=2024-09-16T06:20:50.460786258Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T06:20:50.665 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=sqlstore.transactions t=2024-09-16T06:20:50.506175442Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T06:20:50.666 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a[71651]: logger=sqlstore.transactions t=2024-09-16T06:20:50.51672871Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T06:20:50.985 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:50 smithi190 ceph-mon[25888]: Deploying daemon node-exporter.a on smithi190 2024-09-16T06:20:50.985 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:50 smithi190 ceph-mon[25888]: from='client.? 172.21.15.190:0/2781468473' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T06:20:50.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:50 smithi190 ceph-mon[39731]: Deploying daemon node-exporter.a on smithi190 2024-09-16T06:20:50.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:50 smithi190 ceph-mon[39731]: from='client.? 172.21.15.190:0/2781468473' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T06:20:51.006 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T06:20:51.006 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T06:20:51.006 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T06:20:51.016 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi190.front.sepia.ceph.com 2024-09-16T06:20:51.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-16T06:20:51.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-mon[30578]: Deploying daemon node-exporter.a on smithi190 2024-09-16T06:20:51.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:50 smithi196 ceph-mon[30578]: from='client.? 172.21.15.190:0/2781468473' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T06:20:51.229 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:51.343 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:51 smithi190 bash[85965]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T06:20:51.596 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:51 smithi190 bash[85965]: Getting image source signatures 2024-09-16T06:20:51.597 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:51 smithi190 bash[85965]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T06:20:51.597 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:51 smithi190 bash[85965]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T06:20:51.597 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:51 smithi190 bash[85965]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T06:20:52.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:51 smithi196 ceph-mon[30578]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:52.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:51 smithi190 ceph-mon[25888]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:52.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:51 smithi190 ceph-mon[39731]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:53.307 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:53 smithi190 bash[85965]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T06:20:53.596 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:53 smithi190 bash[85965]: Writing manifest to image destination 2024-09-16T06:20:53.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:53 smithi196 ceph-mon[30578]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:53.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:53 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:53.766 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:20:53 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:20:53.824 INFO:teuthology.orchestra.run.smithi190.stdout:167 167 2024-09-16T06:20:54.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:53 smithi190 ceph-mon[25888]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:54.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:53 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:53 smithi190 ceph-mon[39731]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:54.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:53 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.097 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:53 smithi190 podman[85965]: 2024-09-16 06:20:53.849856497 +0000 UTC m=+2.759029736 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 podman[85965]: 2024-09-16 06:20:54.207624198 +0000 UTC m=+3.116797438 container create 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 podman[85965]: 2024-09-16 06:20:54.434879255 +0000 UTC m=+3.344052494 container init 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 podman[85965]: 2024-09-16 06:20:54.43974214 +0000 UTC m=+3.348915387 container start 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.443Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.443Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T06:20:54.518 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T06:20:54.519 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.444Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T06:20:54.520 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T06:20:54.521 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T06:20:54.522 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T06:20:54.523 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T06:20:54.523 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T06:20:54.523 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a[86201]: ts=2024-09-16T06:20:54.445Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T06:20:54.523 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 bash[85965]: 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c 2024-09-16T06:20:54.846 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:20:54 smithi190 systemd[1]: Started Ceph node-exporter.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:54.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:54 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:55.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:54 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:20:55.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:54 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:55.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:54 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:55.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:54 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:55.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T06:20:55.264 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:55.508 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:55 smithi196 systemd[1]: Starting Ceph node-exporter.b for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:20:55.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:55 smithi196 ceph-mon[30578]: Deploying daemon node-exporter.b on smithi196 2024-09-16T06:20:55.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:55 smithi196 ceph-mon[30578]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:55.765 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:55 smithi196 bash[71922]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T06:20:56.053 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:55 smithi190 ceph-mon[39731]: Deploying daemon node-exporter.b on smithi196 2024-09-16T06:20:56.053 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:55 smithi190 ceph-mon[39731]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:56.054 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:55 smithi190 ceph-mon[25888]: Deploying daemon node-exporter.b on smithi196 2024-09-16T06:20:56.054 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:55 smithi190 ceph-mon[25888]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:20:56.265 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:55 smithi196 bash[71922]: Getting image source signatures 2024-09-16T06:20:56.265 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:55 smithi196 bash[71922]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T06:20:56.265 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:55 smithi196 bash[71922]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T06:20:56.265 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:55 smithi196 bash[71922]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T06:20:57.316 INFO:teuthology.orchestra.run.smithi190.stdout:Backend: cephadm 2024-09-16T06:20:57.317 INFO:teuthology.orchestra.run.smithi190.stdout:Available: Yes 2024-09-16T06:20:57.317 INFO:teuthology.orchestra.run.smithi190.stdout:Paused: No 2024-09-16T06:20:57.536 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:57 smithi196 bash[71922]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T06:20:57.537 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:57 smithi196 bash[71922]: Writing manifest to image destination 2024-09-16T06:20:57.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:57 smithi196 ceph-mon[30578]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:57.849 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:57 smithi196 podman[71922]: 2024-09-16 06:20:57.717599766 +0000 UTC m=+2.303696763 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T06:20:57.849 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:57 smithi196 podman[71922]: 2024-09-16 06:20:57.849603558 +0000 UTC m=+2.435700559 container create 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T06:20:57.913 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:57 smithi190 ceph-mon[25888]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:57.914 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:57 smithi190 ceph-mon[39731]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:57.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T06:20:58.130 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:20:58.143 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 podman[71922]: 2024-09-16 06:20:58.043535245 +0000 UTC m=+2.629632244 container init 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T06:20:58.143 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 podman[71922]: 2024-09-16 06:20:58.047996345 +0000 UTC m=+2.634093341 container start 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T06:20:58.143 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.051Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.051Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.052Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.052Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T06:20:58.144 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T06:20:58.145 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T06:20:58.146 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T06:20:58.146 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T06:20:58.146 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T06:20:58.147 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T06:20:58.147 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T06:20:58.147 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T06:20:58.147 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T06:20:58.147 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T06:20:58.147 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T06:20:58.148 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T06:20:58.148 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T06:20:58.148 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T06:20:58.148 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T06:20:58.148 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T06:20:58.149 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.053Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T06:20:58.150 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.054Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T06:20:58.150 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b[71997]: ts=2024-09-16T06:20:58.054Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T06:20:58.150 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 bash[71922]: 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 2024-09-16T06:20:58.250 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:20:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:20:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:20:58.515 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:20:58 smithi196 systemd[1]: Started Ceph node-exporter.b for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:20:58.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.848 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:20:58.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-09-16T06:20:58.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-16T06:20:58.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T06:20:58.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T06:20:58.849 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:58 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:20:58.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.990 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T06:20:58.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:58 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:20:59.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:20:59.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:20:59.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-09-16T06:20:59.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-16T06:20:59.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T06:20:59.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-16T06:20:59.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:20:59 smithi196 ceph-mon[30578]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T06:21:00.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:00.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[39731]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-16T06:21:00.044 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T06:21:00.045 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-16T06:21:00.045 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:20:59 smithi190 ceph-mon[25888]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:alertmanager.a smithi190 *:9093,9094 starting - - - - 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:grafana.a smithi196 *:3000 starting - - - - 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:iscsi.iscsi.a smithi196 *:5000 running (87s) 56s ago 86s 54.4M - 3.8 67ddd33fa72f cc18aab47149 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:mgr.x smithi196 *:8443,8765 running (5m) 56s ago 5m 435M - 18.2.4-845-ge4e227b3 67ddd33fa72f 33449789fc6c 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:mgr.y smithi190 *:9283,8765 running (6m) 58s ago 6m 456M - 18.2.4-845-ge4e227b3 67ddd33fa72f 057fa89f58e3 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:mon.a smithi190 running (6m) 58s ago 6m 48.1M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f af80c5e1fcd6 2024-09-16T06:21:00.045 INFO:teuthology.orchestra.run.smithi190.stdout:mon.b smithi196 running (5m) 56s ago 5m 47.4M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f 7dcb6335223a 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:mon.c smithi190 running (5m) 58s ago 5m 49.1M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f 4b5276df9a12 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:node-exporter.a smithi190 *:9100 starting - - - - 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:node-exporter.b smithi196 *:9100 starting - - - - 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.0 smithi190 running (4m) 58s ago 4m 45.5M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 57b08e2f5edf 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.1 smithi190 running (4m) 58s ago 4m 47.0M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 2a5f6132de18 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.2 smithi190 running (3m) 58s ago 3m 43.2M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f b1587935d344 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.3 smithi190 running (3m) 58s ago 3m 54.3M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f ad1f0236fde1 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.4 smithi196 running (3m) 56s ago 3m 45.3M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 459bdef3befa 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.5 smithi196 running (2m) 56s ago 2m 45.1M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f b7fdcb8b4666 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.6 smithi196 running (2m) 56s ago 2m 53.3M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f e33d038eadc9 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:osd.7 smithi196 running (117s) 56s ago 117s 44.3M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f e175d40d4637 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:prometheus.a smithi196 *:9095 running (67s) 56s ago 67s 26.1M - 2.43.0 a07b618ecd1d e9742c683e1d 2024-09-16T06:21:00.046 INFO:teuthology.orchestra.run.smithi190.stdout:rgw.foo.a smithi190 *:80 running (100s) 58s ago 100s 76.4M - 18.2.4-845-ge4e227b3 67ddd33fa72f 9bb2fcabd973 2024-09-16T06:21:00.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T06:21:01.220 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:21:01.345 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:01 smithi190 ceph-mon[25888]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:01.345 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:01 smithi190 ceph-mon[25888]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T06:21:01.345 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:01 smithi190 ceph-mon[25888]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:21:01.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:01 smithi190 ceph-mon[39731]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:01.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:01 smithi190 ceph-mon[39731]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T06:21:01.345 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:01 smithi190 ceph-mon[39731]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:21:01.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:01 smithi196 ceph-mon[30578]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:01.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:01 smithi196 ceph-mon[30578]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T06:21:01.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:01 smithi196 ceph-mon[30578]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T06:21:02.340 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:02 smithi190 ceph-mon[39731]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T06:21:02.341 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:02 smithi190 ceph-mon[25888]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T06:21:02.515 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:02 smithi196 ceph-mon[30578]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T06:21:03.012 INFO:teuthology.orchestra.run.smithi190.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T06:21:03.012 INFO:teuthology.orchestra.run.smithi190.stdout:alertmanager ?:9093,9094 0/1 - 83s smithi190=a;count:1 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:grafana ?:3000 1/1 0s ago 80s smithi196=a;count:1 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:iscsi.datapool ?:5000 1/1 0s ago 94s smithi196=iscsi.a;count:1 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:mgr 2/2 61s ago 5m smithi190=y;smithi196=x;count:2 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:mon 3/3 61s ago 5m smithi190:172.21.15.190=a;smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:node-exporter ?:9100 1/2 0s ago 87s smithi190=a;smithi196=b;count:2 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:osd 8 61s ago - 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:prometheus ?:9095 1/1 0s ago 90s smithi196=a;count:1 2024-09-16T06:21:03.013 INFO:teuthology.orchestra.run.smithi190.stdout:rgw.foo.a ?:80 1/1 61s ago 102s smithi190=foo.a;count:1 2024-09-16T06:21:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:03 smithi196 ceph-mon[30578]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T06:21:03.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:03.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:03 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:03.766 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:21:03 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:21:03.993 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:03 smithi190 ceph-mon[25888]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T06:21:03.993 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:03.993 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:03 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:03.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:03 smithi190 ceph-mon[39731]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T06:21:03.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:03.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:03 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:04.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T06:21:05.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:04 smithi196 ceph-mon[30578]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:05.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:04 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:21:05.096 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:04 smithi190 ceph-mon[25888]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:05.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:04 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:21:05.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:04 smithi190 ceph-mon[39731]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:05.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:04 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:21:05.186 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: Cluster is now healthy 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:21:05.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: Cluster is now healthy 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:21:05.848 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:05 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:06.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: Cluster is now healthy 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T06:21:06.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:05 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:07.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:06 smithi196 ceph-mon[30578]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T06:21:07.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:06 smithi196 ceph-mon[30578]: Reconfiguring daemon alertmanager.a on smithi190 2024-09-16T06:21:07.096 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:06 smithi190 ceph-mon[39731]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T06:21:07.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:06 smithi190 ceph-mon[39731]: Reconfiguring daemon alertmanager.a on smithi190 2024-09-16T06:21:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:06 smithi190 ceph-mon[25888]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T06:21:07.097 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:06 smithi190 ceph-mon[25888]: Reconfiguring daemon alertmanager.a on smithi190 2024-09-16T06:21:07.846 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:07 smithi190 ceph-mon[25888]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-16T06:21:07.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:07 smithi190 ceph-mon[39731]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-16T06:21:08.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:07 smithi196 ceph-mon[30578]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-16T06:21:08.264 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:07 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:21:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:21:09.561 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:09 smithi190 systemd[1]: Stopping Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:09.840 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:09 smithi190 ceph-mon[25888]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 49 B/s, 1 objects/s recovering 2024-09-16T06:21:09.841 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:09 smithi190 ceph-mon[39731]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 49 B/s, 1 objects/s recovering 2024-09-16T06:21:10.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:09 smithi196 ceph-mon[30578]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 49 B/s, 1 objects/s recovering 2024-09-16T06:21:10.106 INFO:teuthology.orchestra.run.smithi190.stdout:HOST ADDR LABELS STATUS 2024-09-16T06:21:10.107 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190 172.21.15.190 2024-09-16T06:21:10.107 INFO:teuthology.orchestra.run.smithi190.stdout:smithi196 172.21.15.196 2024-09-16T06:21:10.107 INFO:teuthology.orchestra.run.smithi190.stdout:2 hosts in cluster 2024-09-16T06:21:10.587 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:10 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[84649]: ts=2024-09-16T06:21:10.263Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T06:21:10.587 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:10 smithi190 podman[87929]: 2024-09-16 06:21:10.293690787 +0000 UTC m=+0.304214999 container died cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:10.846 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:10 smithi190 podman[87929]: 2024-09-16 06:21:10.778874426 +0000 UTC m=+0.789398673 container cleanup cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:10.847 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:10 smithi190 bash[87929]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T06:21:11.586 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T06:21:11.846 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:11 smithi190 podman[87962]: 2024-09-16 06:21:11.775818646 +0000 UTC m=+1.474035163 container remove cc52c6d2cb024ba51cc5d5ec2f235c12b873a09a9ac0d5a2772d00f2039dbe1f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:11.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:11 smithi190 ceph-mon[25888]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:11.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:11 smithi190 ceph-mon[25888]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 0 op/s; 45 B/s, 1 objects/s recovering 2024-09-16T06:21:11.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:11 smithi190 ceph-mon[39731]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:11.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:11 smithi190 ceph-mon[39731]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 0 op/s; 45 B/s, 1 objects/s recovering 2024-09-16T06:21:11.978 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:21:12.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:11 smithi196 ceph-mon[30578]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:12.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:11 smithi196 ceph-mon[30578]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 0 op/s; 45 B/s, 1 objects/s recovering 2024-09-16T06:21:12.236 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:11 smithi190 podman[87962]: 2024-09-16 06:21:11.893107857 +0000 UTC m=+1.591324330 volume remove f3b089a94b884ca9c7a4a35a574c66675817ea921ec67b1c8bc48b506d6ec4f8 2024-09-16T06:21:12.597 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:12 smithi190 podman[88046]: 2024-09-16 06:21:12.463319825 +0000 UTC m=+0.513878990 container died c3ae326697ad4713ab554a3ee62fbbaaa9a08a1c1eb0ac93bf16c5364b4e088c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=eloquent_mahavira, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T06:21:13.097 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:12 smithi190 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T06:21:13.097 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:12 smithi190 systemd[1]: Stopped Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:21:13.097 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:12 smithi190 systemd[1]: Starting Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:13.654 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 podman[88201]: 2024-09-16 06:21:13.367057855 +0000 UTC m=+0.196211470 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-16T06:21:13.654 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 podman[88201]: 2024-09-16 06:21:13.470514657 +0000 UTC m=+0.299668261 volume create 17d006d4de38c084fcb3a7f4699d96821edf1049e416d8db45dfacf4b97b25ab 2024-09-16T06:21:13.654 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 podman[88201]: 2024-09-16 06:21:13.579297739 +0000 UTC m=+0.408451338 container create dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:13.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:13 smithi196 ceph-mon[30578]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 38 B/s, 1 objects/s recovering 2024-09-16T06:21:13.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:21:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:21:13.907 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-mon[25888]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 38 B/s, 1 objects/s recovering 2024-09-16T06:21:13.907 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-mon[39731]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 38 B/s, 1 objects/s recovering 2024-09-16T06:21:14.161 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 podman[88201]: 2024-09-16 06:21:13.907560413 +0000 UTC m=+0.736714021 container init dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:14.161 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 podman[88201]: 2024-09-16 06:21:13.913148402 +0000 UTC m=+0.742302002 container start dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:14.161 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.925Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.926Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.926Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.190 port=9094 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.927Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.951Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.951Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.954Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:13 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:13.954Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:14 smithi190 bash[88201]: dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d 2024-09-16T06:21:14.162 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:14 smithi190 systemd[1]: Started Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:21:14.427 INFO:teuthology.orchestra.run.smithi190.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190 /dev/nvme1n1 ssd Linux_82855bf38a6bbf57e11a 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190 /dev/nvme2n1 ssd Linux_88f2efbd2595ffef68c9 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190 /dev/nvme3n1 ssd Linux_185c1f0867c6621424b6 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190 /dev/nvme4n1 ssd Linux_0981ec7af37c38d54881 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi196 /dev/nvme1n1 ssd Linux_9de659ba9459ef1eec13 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi196 /dev/nvme2n1 ssd Linux_f507015bf10bb318bc61 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi196 /dev/nvme3n1 ssd Linux_78bbd8eda7b9cc78ac1e 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.428 INFO:teuthology.orchestra.run.smithi190.stdout:smithi196 /dev/nvme4n1 ssd Linux_6a05bd32c428f2e4cf4c 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[25888]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[25888]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[39731]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:21:14.787 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:14.788 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:14 smithi190 ceph-mon[39731]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:14.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:14 smithi196 ceph-mon[30578]: from='client.24425 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T06:21:14.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:14 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T06:21:14.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:14 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:14.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:14 smithi196 ceph-mon[30578]: from='mgr.24461 172.21.15.190:0/3207323703' entity='mgr.y' 2024-09-16T06:21:15.160 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-16T06:21:15.357 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[25888]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[25888]: Reconfiguring daemon prometheus.a on smithi196 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[25888]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[25888]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 38 B/s, 1 objects/s recovering 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[39731]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[39731]: Reconfiguring daemon prometheus.a on smithi196 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[39731]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:15.758 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-mon[39731]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 38 B/s, 1 objects/s recovering 2024-09-16T06:21:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:15 smithi196 ceph-mon[30578]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T06:21:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:15 smithi196 ceph-mon[30578]: Reconfiguring daemon prometheus.a on smithi196 2024-09-16T06:21:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:15 smithi196 ceph-mon[30578]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T06:21:15.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:15 smithi196 ceph-mon[30578]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 38 B/s, 1 objects/s recovering 2024-09-16T06:21:16.096 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:15 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:15.928Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000248493s 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: alertmanager 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: alertmanager 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: count: 1 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi190=a 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:19:39.125981Z' 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: ports: 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: - 9093 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: - 9094 2024-09-16T06:21:17.230 INFO:teuthology.orchestra.run.smithi190.stdout: running: 0 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout: size: 1 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:events: 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:- 2024-09-16T06:20:35.894167Z service:alertmanager [INFO] "service was created" 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: grafana 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: grafana 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout: count: 1 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi196=a 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout:spec: 2024-09-16T06:21:17.231 INFO:teuthology.orchestra.run.smithi190.stdout: anonymous_access: true 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: protocol: https 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:19:42.607808Z' 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.628498Z' 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: ports: 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: - 3000 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: running: 1 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout: size: 1 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:events: 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:- 2024-09-16T06:20:49.635102Z service:grafana [INFO] "service was created" 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: iscsi 2024-09-16T06:21:17.232 INFO:teuthology.orchestra.run.smithi190.stdout:service_id: datapool 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: iscsi.datapool 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: count: 1 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi196=iscsi.a 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout:spec: 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: api_password: admin 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: api_port: 5000 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: api_user: admin 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: pool: datapool 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: trusted_ip_list: 172.21.15.196 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:19:28.071296Z' 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.628369Z' 2024-09-16T06:21:17.233 INFO:teuthology.orchestra.run.smithi190.stdout: ports: 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: - 5000 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: running: 1 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: size: 1 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: mgr 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: mgr 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: count: 2 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi190=y 2024-09-16T06:21:17.234 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi196=x 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:15:50.475918Z' 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.628056Z' 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: running: 2 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: size: 2 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: mon 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: mon 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: count: 3 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi190:172.21.15.190=a 2024-09-16T06:21:17.235 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi190:[v2:172.21.15.190:3301,v1:172.21.15.190:6790]=c 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi196:172.21.15.196=b 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:15:22.700751Z' 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.627959Z' 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: running: 3 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: size: 3 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: node-exporter 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: node-exporter 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: count: 2 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi190=a 2024-09-16T06:21:17.236 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi196=b 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:19:35.074812Z' 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.628568Z' 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: ports: 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: - 9100 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: running: 2 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout: size: 2 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:events: 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:- 2024-09-16T06:20:58.219514Z service:node-exporter [INFO] "service was created" 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: osd 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: osd 2024-09-16T06:21:17.237 INFO:teuthology.orchestra.run.smithi190.stdout:unmanaged: true 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout:spec: 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: filter_logic: AND 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: objectstore: bluestore 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: container_image_id: 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.628137Z' 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: running: 8 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: size: 8 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: prometheus 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: prometheus 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.238 INFO:teuthology.orchestra.run.smithi190.stdout: count: 1 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi196=a 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:19:32.106769Z' 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:02.628434Z' 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: ports: 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: - 9095 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: running: 1 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout: size: 1 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout:events: 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout:- 2024-09-16T06:20:58.225449Z service:prometheus [INFO] "service was created" 2024-09-16T06:21:17.239 INFO:teuthology.orchestra.run.smithi190.stdout:--- 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout:service_type: rgw 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout:service_id: foo.a 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout:service_name: rgw.foo.a 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout:placement: 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: count: 1 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: hosts: 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: - smithi190=foo.a 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout:status: 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: created: '2024-09-16T06:19:20.024356Z' 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: last_refresh: '2024-09-16T06:21:05.146320Z' 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: ports: 2024-09-16T06:21:17.240 INFO:teuthology.orchestra.run.smithi190.stdout: - 80 2024-09-16T06:21:17.241 INFO:teuthology.orchestra.run.smithi190.stdout: running: 1 2024-09-16T06:21:17.241 INFO:teuthology.orchestra.run.smithi190.stdout: size: 1 2024-09-16T06:21:17.846 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:17 smithi190 ceph-mon[25888]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:17.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:17 smithi190 ceph-mon[39731]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:17.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd41205c-73f2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-16T06:21:18.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:17 smithi196 ceph-mon[30578]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:18.131 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T06:21:18.251 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:17 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: ::ffff:172.21.15.196 - - [16/Sep/2024:06:21:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T06:21:18.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:18 smithi190 ceph-mon[25888]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T06:21:18.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:18 smithi190 ceph-mon[39731]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T06:21:18.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:18 smithi196 ceph-mon[30578]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T06:21:19.515 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 systemd[1]: Stopping Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:19.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:19 smithi190 ceph-mon[25888]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:19.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:19 smithi190 ceph-mon[39731]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:19.918 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-mon[30578]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.636Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T06:21:19.919 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.637Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T06:21:19.920 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.637Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T06:21:19.920 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.637Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T06:21:19.920 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[69703]: ts=2024-09-16T06:21:19.637Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T06:21:19.920 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:19 smithi196 podman[72947]: 2024-09-16 06:21:19.647004586 +0000 UTC m=+0.134973807 container died e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:19.932 INFO:teuthology.orchestra.run.smithi190.stdout:osd 8 17s ago - 2024-09-16T06:21:20.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 podman[72947]: 2024-09-16 06:21:20.001366911 +0000 UTC m=+0.489336174 container cleanup e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:20.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 bash[72947]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T06:21:20.690 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 podman[72965]: 2024-09-16 06:21:20.329515861 +0000 UTC m=+0.679711482 container remove e9742c683e1d60dd99d936373a174ed9aada1de791884f67fa09642e818288a4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:20.690 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T06:21:20.690 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 systemd[1]: Stopped Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:21:20.690 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.177s CPU time. 2024-09-16T06:21:20.690 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 systemd[1]: Starting Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:20.717 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T06:21:21.034 INFO:tasks.cephadm:Teardown begin 2024-09-16T06:21:21.035 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:21:21.036 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 podman[73061]: 2024-09-16 06:21:20.593643546 +0000 UTC m=+0.016313664 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T06:21:21.036 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 podman[73061]: 2024-09-16 06:21:20.725039878 +0000 UTC m=+0.147710042 container create 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:21.036 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 podman[73061]: 2024-09-16 06:21:20.981011937 +0000 UTC m=+0.403682055 container init 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:21.063 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:21:21.092 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T06:21:21.092 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T06:21:21.120 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T06:21:21.149 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T06:21:21.149 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T06:21:21.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a 2024-09-16T06:21:21.200 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE Bus STOPPING 2024-09-16T06:21:21.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:20 smithi196 podman[73061]: 2024-09-16 06:21:20.985677731 +0000 UTC m=+0.408347847 container start 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:21:21.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.009Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T06:21:21.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.009Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T06:21:21.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.009Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi196 (none))" 2024-09-16T06:21:21.266 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.009Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.009Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.011Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.011Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.012Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.012Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.014Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.014Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.554µs 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.014Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.015Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.015Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.015Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.883µs wal_replay_duration=1.47364ms wbl_replay_duration=109ns total_replay_duration=1.508262ms 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.016Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T06:21:21.267 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.016Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T06:21:21.268 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.016Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T06:21:21.268 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.037Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.331661ms db_storage=777ns remote_storage=996ns web_handler=358ns query_engine=634ns scrape=442.681µs scrape_sd=112.207µs notify=14.066µs notify_sd=25.637µs rules=20.445721ms tracing=6.638µs 2024-09-16T06:21:21.268 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.037Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T06:21:21.268 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:21:21.037Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T06:21:21.268 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 bash[73061]: 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e 2024-09-16T06:21:21.268 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:21:21 smithi196 systemd[1]: Started Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:21:21.510 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:21 smithi190 systemd[1]: Stopping Ceph mon.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a[25865]: 2024-09-16T06:21:21.508+0000 7f4ec8669640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a[25865]: 2024-09-16T06:21:21.508+0000 7f4ec8669640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:21 smithi190 podman[88885]: 2024-09-16 06:21:21.631168749 +0000 UTC m=+0.255101244 container died af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE Bus STOPPED 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE Bus STARTING 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE Serving on http://:::9283 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE Bus STARTED 2024-09-16T06:21:21.847 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:21 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:21] ENGINE Bus STOPPING 2024-09-16T06:21:22.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:22 smithi190 podman[88885]: 2024-09-16 06:21:22.074095734 +0000 UTC m=+0.698028264 container cleanup af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T06:21:22.252 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:22 smithi190 bash[88885]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mon.a.smithi190.stdout:Sep 16 06:21:22 smithi190 podman[88955]: 2024-09-16 06:21:22.350482589 +0000 UTC m=+0.721066430 container remove af80c5e1fcd6b3d40bc29771c099528cdd8d7beb91b83fc17a0bf10226d97320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STOPPED 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STARTING 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Serving on http://:::9283 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STARTED 2024-09-16T06:21:22.597 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STOPPING 2024-09-16T06:21:22.824 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T06:21:22.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:22.929 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T06:21:22.930 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-16T06:21:22.930 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.c 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:22 smithi190 systemd[1]: Stopping Ceph mon.c for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STOPPED 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STARTING 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Serving on http://:::9283 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STARTED 2024-09-16T06:21:23.097 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:22 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:22] ENGINE Bus STOPPING 2024-09-16T06:21:23.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:21:23 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug there is no tcmu-runner data available 2024-09-16T06:21:23.845 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T06:21:23.845 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:23] ENGINE Bus STOPPED 2024-09-16T06:21:23.845 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:23] ENGINE Bus STARTING 2024-09-16T06:21:23.845 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:23] ENGINE Serving on http://:::9283 2024-09-16T06:21:23.845 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: [16/Sep/2024:06:21:23] ENGINE Bus STARTED 2024-09-16T06:21:23.845 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c[39708]: 2024-09-16T06:21:23.429+0000 7fdb35e8e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:23.846 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c[39708]: 2024-09-16T06:21:23.429+0000 7fdb35e8e640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T06:21:23.846 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:23 smithi190 podman[89254]: 2024-09-16 06:21:23.511233379 +0000 UTC m=+0.195087537 container died 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:21:24.097 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:21:23 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:21:23.929Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001677349s 2024-09-16T06:21:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:24 smithi190 podman[89254]: 2024-09-16 06:21:24.028021979 +0000 UTC m=+0.711876136 container cleanup 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T06:21:24.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:24 smithi190 bash[89254]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c 2024-09-16T06:21:24.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Sep 16 06:21:24 smithi190 podman[89343]: 2024-09-16 06:21:24.212983278 +0000 UTC m=+0.699453072 container remove 4b5276df9a128da165304f95f8046408d2db15f762453bd8a746628770620791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T06:21:24.734 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T06:21:24.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:24.773 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-16T06:21:24.773 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-16T06:21:24.773 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.b 2024-09-16T06:21:25.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:24 smithi196 systemd[1]: Stopping Ceph mon.b for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:25.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:25 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b[30555]: 2024-09-16T06:21:25.478+0000 7fc7d8ae0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:25.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:25 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b[30555]: 2024-09-16T06:21:25.478+0000 7fc7d8ae0640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T06:21:25.765 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:25 smithi196 podman[73775]: 2024-09-16 06:21:25.540583967 +0000 UTC m=+0.523841535 container died 7dcb6335223adb6ece404b21bdbce2ed613bef1f933455f5b9443f3e02ab9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T06:21:26.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:25 smithi196 podman[73775]: 2024-09-16 06:21:25.897412121 +0000 UTC m=+0.880669689 container cleanup 7dcb6335223adb6ece404b21bdbce2ed613bef1f933455f5b9443f3e02ab9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T06:21:26.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 16 06:21:25 smithi196 bash[73775]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mon-b 2024-09-16T06:21:26.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T06:21:26.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:26.421 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-16T06:21:26.421 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T06:21:26.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y 2024-09-16T06:21:26.716 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:26 smithi190 systemd[1]: Stopping Ceph mgr.y for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:26.716 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:26 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y[26142]: 2024-09-16T06:21:26.690+0000 7fe51ba8e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:26.990 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:26 smithi190 podman[89611]: 2024-09-16 06:21:26.721712024 +0000 UTC m=+0.118045037 container died 057fa89f58e3102a32969b98536b3511a9fa8d25243e1282161f3c3904bf5a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:21:27.346 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:27 smithi190 podman[89611]: 2024-09-16 06:21:27.10485829 +0000 UTC m=+0.501191302 container cleanup 057fa89f58e3102a32969b98536b3511a9fa8d25243e1282161f3c3904bf5a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b) 2024-09-16T06:21:27.346 INFO:journalctl@ceph.mgr.y.smithi190.stdout:Sep 16 06:21:27 smithi190 bash[89611]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-y 2024-09-16T06:21:27.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T06:21:27.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:27.580 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T06:21:27.580 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T06:21:27.580 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.x 2024-09-16T06:21:27.912 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:21:27 smithi196 systemd[1]: Stopping Ceph mgr.x for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:28.166 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:21:27 smithi196 podman[73909]: 2024-09-16 06:21:27.912322446 +0000 UTC m=+0.144178232 container died 33449789fc6c3fd25bb46a33cb024ea2f1800659b4b4733cb9bf444f84716b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T06:21:28.515 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:21:28 smithi196 podman[73909]: 2024-09-16 06:21:28.272936706 +0000 UTC m=+0.504792491 container cleanup 33449789fc6c3fd25bb46a33cb024ea2f1800659b4b4733cb9bf444f84716b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T06:21:28.515 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 16 06:21:28 smithi196 bash[73909]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-mgr-x 2024-09-16T06:21:28.705 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T06:21:28.740 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:28.741 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T06:21:28.741 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T06:21:28.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.0 2024-09-16T06:21:29.059 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:28 smithi190 systemd[1]: Stopping Ceph osd.0 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:29.346 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0[43985]: 2024-09-16T06:21:29.059+0000 7f87e64ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:29.347 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0[43985]: 2024-09-16T06:21:29.059+0000 7f87e64ed640 -1 osd.0 72 *** Got signal Terminated *** 2024-09-16T06:21:29.347 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:29 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0[43985]: 2024-09-16T06:21:29.059+0000 7f87e64ed640 -1 osd.0 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:21:34.710 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:34 smithi190 podman[89745]: 2024-09-16 06:21:34.415126004 +0000 UTC m=+5.477431098 container died 57b08e2f5edf8d042cf3b1d8cac36dcf38d0fd073ebf81c6cbf5bc7557bfbfa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0) 2024-09-16T06:21:35.097 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:34 smithi190 podman[89745]: 2024-09-16 06:21:34.900893169 +0000 UTC m=+5.963198302 container cleanup 57b08e2f5edf8d042cf3b1d8cac36dcf38d0fd073ebf81c6cbf5bc7557bfbfa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T06:21:35.097 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:34 smithi190 bash[89745]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0 2024-09-16T06:21:35.512 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89764]: 2024-09-16 06:21:35.152326558 +0000 UTC m=+0.732075881 container remove 57b08e2f5edf8d042cf3b1d8cac36dcf38d0fd073ebf81c6cbf5bc7557bfbfa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T06:21:35.822 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89856]: 2024-09-16 06:21:35.416001385 +0000 UTC m=+0.016098175 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:21:35.823 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89856]: 2024-09-16 06:21:35.553222423 +0000 UTC m=+0.153319255 container create 6a6f5f194835eca5f3b7a3364b5fc452548ff0eecbac133b60f2cbd52e31a2a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T06:21:35.823 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89856]: 2024-09-16 06:21:35.755968158 +0000 UTC m=+0.356064949 container init 6a6f5f194835eca5f3b7a3364b5fc452548ff0eecbac133b60f2cbd52e31a2a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T06:21:35.823 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89856]: 2024-09-16 06:21:35.761114643 +0000 UTC m=+0.361211433 container start 6a6f5f194835eca5f3b7a3364b5fc452548ff0eecbac133b60f2cbd52e31a2a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:21:35.823 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89856]: 2024-09-16 06:21:35.822352381 +0000 UTC m=+0.422449189 container attach 6a6f5f194835eca5f3b7a3364b5fc452548ff0eecbac133b60f2cbd52e31a2a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T06:21:36.097 INFO:journalctl@ceph.osd.0.smithi190.stdout:Sep 16 06:21:35 smithi190 podman[89856]: 2024-09-16 06:21:35.956484719 +0000 UTC m=+0.556581526 container died 6a6f5f194835eca5f3b7a3364b5fc452548ff0eecbac133b60f2cbd52e31a2a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T06:21:36.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T06:21:36.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:36.623 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T06:21:36.623 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T06:21:36.623 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.1 2024-09-16T06:21:36.846 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:36 smithi190 systemd[1]: Stopping Ceph osd.1 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:37.346 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1[52202]: 2024-09-16T06:21:36.974+0000 7f26c6fe1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:37.346 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1[52202]: 2024-09-16T06:21:36.974+0000 7f26c6fe1640 -1 osd.1 72 *** Got signal Terminated *** 2024-09-16T06:21:37.347 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:36 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1[52202]: 2024-09-16T06:21:36.974+0000 7f26c6fe1640 -1 osd.1 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:21:42.814 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:42 smithi190 podman[89987]: 2024-09-16 06:21:42.444393636 +0000 UTC m=+5.591394299 container died 2a5f6132de18b2103f3ab060537fcde7316c62ddc1d6aba018bfe52cca6508cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=reef-e4e227b) 2024-09-16T06:21:43.096 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:42 smithi190 podman[89987]: 2024-09-16 06:21:42.8138656 +0000 UTC m=+5.960866301 container cleanup 2a5f6132de18b2103f3ab060537fcde7316c62ddc1d6aba018bfe52cca6508cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T06:21:43.097 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:42 smithi190 bash[89987]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1 2024-09-16T06:21:43.429 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90010]: 2024-09-16 06:21:43.106993214 +0000 UTC m=+0.663168922 container remove 2a5f6132de18b2103f3ab060537fcde7316c62ddc1d6aba018bfe52cca6508cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T06:21:43.684 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90105]: 2024-09-16 06:21:43.332276022 +0000 UTC m=+0.015760965 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:21:43.685 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90105]: 2024-09-16 06:21:43.457763897 +0000 UTC m=+0.141248871 container create 5315e1e62bd4ac573d41a040cb94074d1784f72ce42f7cc90c5a619f26b798da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T06:21:43.685 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90105]: 2024-09-16 06:21:43.685319826 +0000 UTC m=+0.368804770 container init 5315e1e62bd4ac573d41a040cb94074d1784f72ce42f7cc90c5a619f26b798da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T06:21:44.097 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90105]: 2024-09-16 06:21:43.690407204 +0000 UTC m=+0.373892132 container start 5315e1e62bd4ac573d41a040cb94074d1784f72ce42f7cc90c5a619f26b798da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-16T06:21:44.097 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90105]: 2024-09-16 06:21:43.768445165 +0000 UTC m=+0.451930122 container attach 5315e1e62bd4ac573d41a040cb94074d1784f72ce42f7cc90c5a619f26b798da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-16T06:21:44.097 INFO:journalctl@ceph.osd.1.smithi190.stdout:Sep 16 06:21:43 smithi190 podman[90105]: 2024-09-16 06:21:43.878655761 +0000 UTC m=+0.562140689 container died 5315e1e62bd4ac573d41a040cb94074d1784f72ce42f7cc90c5a619f26b798da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T06:21:44.521 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T06:21:44.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:44.565 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T06:21:44.565 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T06:21:44.565 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.2 2024-09-16T06:21:44.847 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:44 smithi190 systemd[1]: Stopping Ceph osd.2 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:45.346 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:44 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2[60754]: 2024-09-16T06:21:44.910+0000 7f10f7807640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:45.346 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:44 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2[60754]: 2024-09-16T06:21:44.910+0000 7f10f7807640 -1 osd.2 72 *** Got signal Terminated *** 2024-09-16T06:21:45.347 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:44 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2[60754]: 2024-09-16T06:21:44.910+0000 7f10f7807640 -1 osd.2 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:21:50.636 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:50 smithi190 podman[90231]: 2024-09-16 06:21:50.373649614 +0000 UTC m=+5.574305186 container died b1587935d344ef40a91881b4314f8876ac944a065fbecb879fff549765beaaec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T06:21:51.011 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:50 smithi190 podman[90231]: 2024-09-16 06:21:50.801566635 +0000 UTC m=+6.002222273 container cleanup b1587935d344ef40a91881b4314f8876ac944a065fbecb879fff549765beaaec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T06:21:51.011 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:50 smithi190 bash[90231]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2 2024-09-16T06:21:51.347 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90253]: 2024-09-16 06:21:51.011349012 +0000 UTC m=+0.632161716 container remove b1587935d344ef40a91881b4314f8876ac944a065fbecb879fff549765beaaec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T06:21:51.732 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90347]: 2024-09-16 06:21:51.323861823 +0000 UTC m=+0.016180472 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:21:51.732 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90347]: 2024-09-16 06:21:51.437159668 +0000 UTC m=+0.129478317 container create a394fb4930b31db5f749a7c29c58aa7dd0102f83add3a0c1012e21b75c9a1d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T06:21:51.732 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90347]: 2024-09-16 06:21:51.63979811 +0000 UTC m=+0.332116760 container init a394fb4930b31db5f749a7c29c58aa7dd0102f83add3a0c1012e21b75c9a1d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=reef-e4e227b) 2024-09-16T06:21:51.732 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90347]: 2024-09-16 06:21:51.644848708 +0000 UTC m=+0.337167360 container start a394fb4930b31db5f749a7c29c58aa7dd0102f83add3a0c1012e21b75c9a1d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T06:21:51.732 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90347]: 2024-09-16 06:21:51.731649004 +0000 UTC m=+0.423967671 container attach a394fb4930b31db5f749a7c29c58aa7dd0102f83add3a0c1012e21b75c9a1d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T06:21:52.097 INFO:journalctl@ceph.osd.2.smithi190.stdout:Sep 16 06:21:51 smithi190 podman[90347]: 2024-09-16 06:21:51.838782628 +0000 UTC m=+0.531101280 container died a394fb4930b31db5f749a7c29c58aa7dd0102f83add3a0c1012e21b75c9a1d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T06:21:52.399 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T06:21:52.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:21:52.438 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T06:21:52.439 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T06:21:52.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.3 2024-09-16T06:21:52.649 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:52 smithi190 systemd[1]: Stopping Ceph osd.3 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:21:53.096 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:21:52.789+0000 7f8a7cbdb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:21:53.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:21:52.789+0000 7f8a7cbdb640 -1 osd.3 72 *** Got signal Terminated *** 2024-09-16T06:21:53.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:52 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:21:52.789+0000 7f8a7cbdb640 -1 osd.3 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:21:55.515 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:21:55 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:21:55.190+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:21:56.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:55 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:21:55.778+0000 7f8a751e7640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.993002+0000 front 2024-09-16T06:21:30.993161+0000 (oldest deadline 2024-09-16T06:21:55.093275+0000) 2024-09-16T06:21:56.515 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:21:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:21:56.170+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:21:56.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:21:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:21:56.200+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:21:57.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:56 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:21:56.776+0000 7f8a751e7640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.993002+0000 front 2024-09-16T06:21:30.993161+0000 (oldest deadline 2024-09-16T06:21:55.093275+0000) 2024-09-16T06:21:57.265 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:21:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:21:57.157+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:21:57.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:21:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:21:57.186+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:21:57.266 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:21:56 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:21:56.946+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:21:58.097 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:57 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3[69245]: 2024-09-16T06:21:57.776+0000 7f8a751e7640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.993002+0000 front 2024-09-16T06:21:30.993161+0000 (oldest deadline 2024-09-16T06:21:55.093275+0000) 2024-09-16T06:21:58.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:21:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:21:58.171+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:21:58.205 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:21:57 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:21:57.925+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:21:58.463 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:58 smithi190 podman[90477]: 2024-09-16 06:21:58.188783468 +0000 UTC m=+5.502216766 container died ad1f0236fde1edcbd582f272ffefe52d5558bb4d4bba41baa0e3780ebfc4cda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T06:21:58.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:21:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:21:58.204+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:21:58.846 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:58 smithi190 podman[90477]: 2024-09-16 06:21:58.662722731 +0000 UTC m=+5.976156029 container cleanup ad1f0236fde1edcbd582f272ffefe52d5558bb4d4bba41baa0e3780ebfc4cda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T06:21:58.847 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:58 smithi190 bash[90477]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3 2024-09-16T06:21:59.149 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:21:58 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:21:58.880+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:21:59.228 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:58 smithi190 podman[90508]: 2024-09-16 06:21:58.913968914 +0000 UTC m=+0.718609103 container remove ad1f0236fde1edcbd582f272ffefe52d5558bb4d4bba41baa0e3780ebfc4cda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T06:21:59.517 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:21:59 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:21:59.284+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:21:59.517 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:21:59 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:21:59.239+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:21:59.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:21:59 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:21:59.148+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:21:59.518 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:59 smithi190 podman[90603]: 2024-09-16 06:21:59.130998348 +0000 UTC m=+0.016281517 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:21:59.518 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:59 smithi190 podman[90603]: 2024-09-16 06:21:59.247965959 +0000 UTC m=+0.133249172 container create 38bdc9ca51b278b274f36c752e967ab71a59904181122f49dcfaa4d0790c9539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T06:21:59.518 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:59 smithi190 podman[90603]: 2024-09-16 06:21:59.441917298 +0000 UTC m=+0.327200466 container init 38bdc9ca51b278b274f36c752e967ab71a59904181122f49dcfaa4d0790c9539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T06:21:59.518 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:59 smithi190 podman[90603]: 2024-09-16 06:21:59.446556617 +0000 UTC m=+0.331839785 container start 38bdc9ca51b278b274f36c752e967ab71a59904181122f49dcfaa4d0790c9539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T06:21:59.847 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:59 smithi190 podman[90603]: 2024-09-16 06:21:59.51676942 +0000 UTC m=+0.402052604 container attach 38bdc9ca51b278b274f36c752e967ab71a59904181122f49dcfaa4d0790c9539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T06:21:59.847 INFO:journalctl@ceph.osd.3.smithi190.stdout:Sep 16 06:21:59 smithi190 podman[90603]: 2024-09-16 06:21:59.643297234 +0000 UTC m=+0.528580402 container died 38bdc9ca51b278b274f36c752e967ab71a59904181122f49dcfaa4d0790c9539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T06:22:00.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:00.099+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:22:00.248 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:21:59 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:21:59.851+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:00.331 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T06:22:00.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:00.377 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T06:22:00.377 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T06:22:00.378 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.4 2024-09-16T06:22:00.515 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:00 smithi196 systemd[1]: Stopping Ceph osd.4 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:00.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:00.247+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:00.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:00.279+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:01.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:00.848+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:01.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:00.699+0000 7faa4b068640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:22:01.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:00.699+0000 7faa4b068640 -1 osd.4 72 *** Got signal Terminated *** 2024-09-16T06:22:01.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:00 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:00.699+0000 7faa4b068640 -1 osd.4 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:22:01.515 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:01 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:01.096+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:22:01.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:01 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:01.262+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:01.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:01 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:01.236+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:02.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:01 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:01.855+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:02.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:02 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:02.218+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:02.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:02 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:02.223+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:02.516 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:02 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:02.140+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:22:03.147 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:02 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:02.841+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:03.515 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:03 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:03.147+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:22:03.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:03 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:03.263+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:03.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:03 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:03.188+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:04.100 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:03 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:03.826+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:04.515 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:04 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:04.099+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:22:04.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:04 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:04.164+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:04.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:04 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:04.283+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:04.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:04 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:04.283+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:05.057 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:04 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:04.793+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:05.058 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:04 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:04.793+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:05.327 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:05.163+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:05.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:05.057+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:30.114021+0000 front 2024-09-16T06:21:30.113908+0000 (oldest deadline 2024-09-16T06:21:54.813950+0000) 2024-09-16T06:22:05.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4[35866]: 2024-09-16T06:22:05.057+0000 7faa4767c640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.815554+0000 front 2024-09-16T06:21:41.815594+0000 (oldest deadline 2024-09-16T06:22:04.115453+0000) 2024-09-16T06:22:05.765 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:05.327+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:05.765 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:05.327+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:06.172 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:05.830+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:06.172 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:05 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:05.830+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:06.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:06 smithi196 podman[74054]: 2024-09-16 06:22:06.17817338 +0000 UTC m=+5.603371777 container died 459bdef3befa031c17a1a170ceca3884f3d867710fd42e8ddde27ec50c2f0726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731) 2024-09-16T06:22:06.457 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:06 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:06.170+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:06.457 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:06 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:06.328+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:06.457 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:06 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:06.328+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:06.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:06 smithi196 podman[74054]: 2024-09-16 06:22:06.573026329 +0000 UTC m=+5.998224796 container cleanup 459bdef3befa031c17a1a170ceca3884f3d867710fd42e8ddde27ec50c2f0726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, ceph=True) 2024-09-16T06:22:06.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:06 smithi196 bash[74054]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4 2024-09-16T06:22:07.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:06 smithi196 podman[74075]: 2024-09-16 06:22:06.807313649 +0000 UTC m=+0.620488273 container remove 459bdef3befa031c17a1a170ceca3884f3d867710fd42e8ddde27ec50c2f0726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T06:22:07.144 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:06 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:06.785+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:07.145 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:06 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:06.785+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:07.443 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:07.148+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:07.443 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:07.329+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:07.443 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:07.329+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:07.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:07 smithi196 podman[74169]: 2024-09-16 06:22:07.047110388 +0000 UTC m=+0.015712403 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:22:07.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:07 smithi196 podman[74169]: 2024-09-16 06:22:07.166472267 +0000 UTC m=+0.135074308 container create 156f5290900056ee1d43053895b2b0b76b5defed55f6e816e9a04f2d0024bfbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T06:22:07.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:07 smithi196 podman[74169]: 2024-09-16 06:22:07.360682724 +0000 UTC m=+0.329284723 container init 156f5290900056ee1d43053895b2b0b76b5defed55f6e816e9a04f2d0024bfbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b) 2024-09-16T06:22:07.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:07 smithi196 podman[74169]: 2024-09-16 06:22:07.365806635 +0000 UTC m=+0.334408637 container start 156f5290900056ee1d43053895b2b0b76b5defed55f6e816e9a04f2d0024bfbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, io.buildah.version=1.37.2) 2024-09-16T06:22:07.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:07 smithi196 podman[74169]: 2024-09-16 06:22:07.443776189 +0000 UTC m=+0.412378196 container attach 156f5290900056ee1d43053895b2b0b76b5defed55f6e816e9a04f2d0024bfbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T06:22:07.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 16 06:22:07 smithi196 podman[74169]: 2024-09-16 06:22:07.578328431 +0000 UTC m=+0.546930439 container died 156f5290900056ee1d43053895b2b0b76b5defed55f6e816e9a04f2d0024bfbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T06:22:08.185 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:07.796+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:08.185 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:07 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:07.796+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:08.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T06:22:08.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:08.296 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T06:22:08.296 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T06:22:08.297 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.5 2024-09-16T06:22:08.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:08.361+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:08.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:08.361+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:08.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:08.184+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:08.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:08.184+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.581243+0000 front 2024-09-16T06:21:41.581286+0000 (oldest deadline 2024-09-16T06:22:07.481020+0000) 2024-09-16T06:22:08.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:08 smithi196 systemd[1]: Stopping Ceph osd.5 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:09.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:08.785+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:09.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:08.785+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:09.016 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:08.627+0000 7f78830d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:22:09.016 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:08.627+0000 7f78830d2640 -1 osd.5 72 *** Got signal Terminated *** 2024-09-16T06:22:09.016 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:08 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:08.627+0000 7f78830d2640 -1 osd.5 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:22:09.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:09 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:09.210+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:09.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:09 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:09.210+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.581243+0000 front 2024-09-16T06:21:41.581286+0000 (oldest deadline 2024-09-16T06:22:07.481020+0000) 2024-09-16T06:22:09.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:09 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:09.315+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:09.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:09 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:09.315+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:10.244 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:09 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:09.824+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:10.244 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:09 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:09.824+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:10.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:10 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:10.243+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:10.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:10 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:10.243+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.581243+0000 front 2024-09-16T06:21:41.581286+0000 (oldest deadline 2024-09-16T06:22:07.481020+0000) 2024-09-16T06:22:10.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:10 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:10.271+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:10.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:10 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:10.271+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:11.224 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:10 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:10.861+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:11.224 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:10 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:10.861+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:11.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:11.224+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:11.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:11.224+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.581243+0000 front 2024-09-16T06:21:41.581286+0000 (oldest deadline 2024-09-16T06:22:07.481020+0000) 2024-09-16T06:22:11.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:11.245+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:11.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:11.245+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:12.201 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:11.869+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:12.202 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:11.869+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:12.202 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:11 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:11.869+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:12.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:12.227+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:12.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:12.227+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:12.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:12.198+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:12.516 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:12.198+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.581243+0000 front 2024-09-16T06:21:41.581286+0000 (oldest deadline 2024-09-16T06:22:07.481020+0000) 2024-09-16T06:22:13.177 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:12.840+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:13.177 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:12.840+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:13.177 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:12 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:12.840+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:13.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:13.176+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:32.780340+0000 front 2024-09-16T06:21:32.780351+0000 (oldest deadline 2024-09-16T06:21:58.680235+0000) 2024-09-16T06:22:13.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5[43988]: 2024-09-16T06:22:13.176+0000 7f787f6e6640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.581243+0000 front 2024-09-16T06:21:41.581286+0000 (oldest deadline 2024-09-16T06:22:07.481020+0000) 2024-09-16T06:22:13.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:13.248+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:13.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:13.248+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:14.210 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:14 smithi196 podman[74295]: 2024-09-16 06:22:14.016433329 +0000 UTC m=+5.494245068 container died b7fdcb8b4666e904b3100cc9f809638ed989766416839601a1bd89f288275208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0) 2024-09-16T06:22:14.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:13.857+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:14.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:13.857+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:14.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:13 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:13.857+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:14.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:14 smithi196 podman[74295]: 2024-09-16 06:22:14.399904318 +0000 UTC m=+5.877716057 container cleanup b7fdcb8b4666e904b3100cc9f809638ed989766416839601a1bd89f288275208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T06:22:14.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:14 smithi196 bash[74295]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5 2024-09-16T06:22:14.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:14 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:14.209+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:14.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:14 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:14.209+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:14.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:14 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:14.209+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:14.943 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:14 smithi196 podman[74315]: 2024-09-16 06:22:14.61757992 +0000 UTC m=+0.598673702 container remove b7fdcb8b4666e904b3100cc9f809638ed989766416839601a1bd89f288275208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T06:22:14.944 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:14 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:14.827+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:14.944 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:14 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:14.827+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:14.944 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:14 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:14.827+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:15.253 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:15 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:15.177+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:15.253 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:15 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:15.177+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:15.253 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:15 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:15.177+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:15.253 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:14 smithi196 podman[74407]: 2024-09-16 06:22:14.846754737 +0000 UTC m=+0.016012581 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:22:15.253 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:14 smithi196 podman[74407]: 2024-09-16 06:22:14.951790352 +0000 UTC m=+0.121048245 container create b0362e56f20a30ad6574e6751f3369043dafb8a6b18d17927907b8dd78465ae3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=reef-e4e227b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:22:15.254 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:15 smithi196 podman[74407]: 2024-09-16 06:22:15.145648462 +0000 UTC m=+0.314906306 container init b0362e56f20a30ad6574e6751f3369043dafb8a6b18d17927907b8dd78465ae3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, io.buildah.version=1.37.2) 2024-09-16T06:22:15.254 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:15 smithi196 podman[74407]: 2024-09-16 06:22:15.151166792 +0000 UTC m=+0.320424639 container start b0362e56f20a30ad6574e6751f3369043dafb8a6b18d17927907b8dd78465ae3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:22:15.254 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:15 smithi196 podman[74407]: 2024-09-16 06:22:15.253692188 +0000 UTC m=+0.422950033 container attach b0362e56f20a30ad6574e6751f3369043dafb8a6b18d17927907b8dd78465ae3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:22:15.515 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:15 smithi196 podman[74407]: 2024-09-16 06:22:15.343143004 +0000 UTC m=+0.512400849 container died b0362e56f20a30ad6574e6751f3369043dafb8a6b18d17927907b8dd78465ae3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-16T06:22:15.989 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 16 06:22:15 smithi196 podman[74450]: 2024-09-16 06:22:15.965698625 +0000 UTC m=+0.616908562 container remove b0362e56f20a30ad6574e6751f3369043dafb8a6b18d17927907b8dd78465ae3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:22:15.989 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:15 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:15.850+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:15.990 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:15 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:15.850+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:15.990 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:15 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:15.850+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:15.994 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T06:22:16.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:16.034 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T06:22:16.034 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T06:22:16.034 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.6 2024-09-16T06:22:16.265 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 systemd[1]: Stopping Ceph osd.6 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:16.265 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:16.167+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:16.265 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:16.167+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:16.266 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:16.167+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:16.765 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:16.370+0000 7f07425b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:22:16.765 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:16.370+0000 7f07425b1640 -1 osd.6 72 *** Got signal Terminated *** 2024-09-16T06:22:16.765 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:16.370+0000 7f07425b1640 -1 osd.6 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:22:17.177 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:16.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:17.178 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:16.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:17.178 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:16 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:16.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:17.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:17.177+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:17.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:17.177+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:17.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:17.177+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:18.154 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:17.813+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:18.154 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:17.813+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:18.154 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:17 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:17.813+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:18.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:18 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:18.153+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:18.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:18 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:18.153+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:18.516 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:18 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:18.153+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:19.159 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:18 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:18.802+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:19.160 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:18 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:18.802+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:19.160 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:18 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:18.802+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:19.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:19.159+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:19.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:19.159+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:19.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:19.159+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:20.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:19.756+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:20.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:19.756+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:20.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:19.756+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:20.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:19 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:19.756+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:20.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:20.141+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:20.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:20.141+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:20.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:20.141+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:21.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:20.762+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:21.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:20.762+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:21.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:20.762+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:21.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:20 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:20.762+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:21.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:21.165+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.307365+0000 front 2024-09-16T06:21:31.307388+0000 (oldest deadline 2024-09-16T06:21:55.407244+0000) 2024-09-16T06:22:21.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:21.165+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:41.808704+0000 front 2024-09-16T06:21:41.808725+0000 (oldest deadline 2024-09-16T06:22:04.108565+0000) 2024-09-16T06:22:21.515 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6[52193]: 2024-09-16T06:22:21.165+0000 7f073abbd640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:48.209540+0000 front 2024-09-16T06:21:48.209579+0000 (oldest deadline 2024-09-16T06:22:13.509439+0000) 2024-09-16T06:22:21.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:21.020Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.190:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.190:8765: connect: connection refused" 2024-09-16T06:22:21.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:21.020Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.190:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.190:8765: connect: connection refused" 2024-09-16T06:22:21.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:21.022Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.190:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.190:8765: connect: connection refused" 2024-09-16T06:22:21.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:21.022Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.190:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.190:8765: connect: connection refused" 2024-09-16T06:22:21.516 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:21.022Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.190:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.190:8765: connect: connection refused" 2024-09-16T06:22:22.011 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:21 smithi196 podman[74533]: 2024-09-16 06:22:21.687617434 +0000 UTC m=+5.427929793 container died e33d038eadc9ede95b87b4fa35f50562d03f7d41b5d2b9a62546bc8316c1abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T06:22:22.011 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:21.767+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:22.011 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:21.767+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:22.011 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:21.767+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:22.011 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:21 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:21.767+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:22.265 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:22 smithi196 podman[74533]: 2024-09-16 06:22:22.118047592 +0000 UTC m=+5.858359944 container cleanup e33d038eadc9ede95b87b4fa35f50562d03f7d41b5d2b9a62546bc8316c1abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T06:22:22.266 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:22 smithi196 bash[74533]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6 2024-09-16T06:22:22.765 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:22 smithi196 podman[74557]: 2024-09-16 06:22:22.361001568 +0000 UTC m=+0.665325545 container remove e33d038eadc9ede95b87b4fa35f50562d03f7d41b5d2b9a62546bc8316c1abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:22 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:22.783+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:22 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:22.783+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:22 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:22.783+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:22 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:22.783+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:22 smithi196 podman[74650]: 2024-09-16 06:22:22.680663179 +0000 UTC m=+0.015911083 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:22 smithi196 podman[74650]: 2024-09-16 06:22:22.778472419 +0000 UTC m=+0.113720315 container create 4647c10884ca22f6c1afc2cb0f621ab8f0256e8fd556b16f5618ee5f6ad70ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:22 smithi196 podman[74650]: 2024-09-16 06:22:22.9976633 +0000 UTC m=+0.332911190 container init 4647c10884ca22f6c1afc2cb0f621ab8f0256e8fd556b16f5618ee5f6ad70ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:23 smithi196 podman[74650]: 2024-09-16 06:22:23.002751458 +0000 UTC m=+0.337999349 container start 4647c10884ca22f6c1afc2cb0f621ab8f0256e8fd556b16f5618ee5f6ad70ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T06:22:23.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:23 smithi196 podman[74650]: 2024-09-16 06:22:23.097258101 +0000 UTC m=+0.432505991 container attach 4647c10884ca22f6c1afc2cb0f621ab8f0256e8fd556b16f5618ee5f6ad70ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T06:22:23.509 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 16 06:22:23 smithi196 podman[74650]: 2024-09-16 06:22:23.197112863 +0000 UTC m=+0.532360764 container died 4647c10884ca22f6c1afc2cb0f621ab8f0256e8fd556b16f5618ee5f6ad70ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b) 2024-09-16T06:22:23.808 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T06:22:23.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:23.847 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T06:22:23.847 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T06:22:23.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.7 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:23 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:23.815+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:23 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:23.815+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:23 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:23.815+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:23 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:23.815+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:23 smithi196 systemd[1]: Stopping Ceph osd.7 for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.180+0000 7fa6248a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:22:24.181 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.180+0000 7fa6248a7640 -1 osd.7 72 *** Got signal Terminated *** 2024-09-16T06:22:24.182 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.180+0000 7fa6248a7640 -1 osd.7 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T06:22:25.266 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:25.266 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:25.266 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:25.266 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:24 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:24.811+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:26.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:25 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:25.822+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:26.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:25 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:25.822+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:26.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:25 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:25.822+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:26.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:25 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:25.822+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:27.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:26 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:26.778+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:27.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:26 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:26.778+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:27.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:26 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:26.778+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:27.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:26 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:26.778+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:28.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:27 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:27.765+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:28.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:27 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:27.765+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:28.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:27 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:27.765+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:28.016 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:27 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:27.765+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:29.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:28 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:28.784+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6806 osd.0 since back 2024-09-16T06:21:31.891765+0000 front 2024-09-16T06:21:31.891780+0000 (oldest deadline 2024-09-16T06:21:55.991691+0000) 2024-09-16T06:22:29.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:28 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:28.784+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6814 osd.1 since back 2024-09-16T06:21:39.393376+0000 front 2024-09-16T06:21:39.393363+0000 (oldest deadline 2024-09-16T06:22:03.993816+0000) 2024-09-16T06:22:29.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:28 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:28.784+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6822 osd.2 since back 2024-09-16T06:21:46.294863+0000 front 2024-09-16T06:21:46.294777+0000 (oldest deadline 2024-09-16T06:22:11.594693+0000) 2024-09-16T06:22:29.266 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:28 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7[60235]: 2024-09-16T06:22:28.784+0000 7fa620ebb640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.190:6830 osd.3 since back 2024-09-16T06:21:53.295600+0000 front 2024-09-16T06:21:53.295577+0000 (oldest deadline 2024-09-16T06:22:19.195515+0000) 2024-09-16T06:22:29.765 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:29 smithi196 podman[74777]: 2024-09-16 06:22:29.485493971 +0000 UTC m=+5.415092495 container died e175d40d4637c902c05faff418c807e3b290575e8a7fab7b5ce840d8eb177ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T06:22:30.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:29 smithi196 podman[74777]: 2024-09-16 06:22:29.811152169 +0000 UTC m=+5.740750678 container cleanup e175d40d4637c902c05faff418c807e3b290575e8a7fab7b5ce840d8eb177ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T06:22:30.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:29 smithi196 bash[74777]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7 2024-09-16T06:22:30.265 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74810]: 2024-09-16 06:22:30.020256854 +0000 UTC m=+0.528791321 container remove e175d40d4637c902c05faff418c807e3b290575e8a7fab7b5ce840d8eb177ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T06:22:30.739 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74903]: 2024-09-16 06:22:30.330405915 +0000 UTC m=+0.015857191 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T06:22:30.739 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74903]: 2024-09-16 06:22:30.437797504 +0000 UTC m=+0.123248815 container create 0e684684586cbf80547ad0d34dae759f66881c802173ac67361b250817b1a785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T06:22:30.740 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74903]: 2024-09-16 06:22:30.648296395 +0000 UTC m=+0.333747661 container init 0e684684586cbf80547ad0d34dae759f66881c802173ac67361b250817b1a785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T06:22:30.740 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74903]: 2024-09-16 06:22:30.653405825 +0000 UTC m=+0.338857092 container start 0e684684586cbf80547ad0d34dae759f66881c802173ac67361b250817b1a785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T06:22:30.740 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74903]: 2024-09-16 06:22:30.739649536 +0000 UTC m=+0.425100808 container attach 0e684684586cbf80547ad0d34dae759f66881c802173ac67361b250817b1a785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T06:22:31.015 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 16 06:22:30 smithi196 podman[74903]: 2024-09-16 06:22:30.84447841 +0000 UTC m=+0.529929685 container died 0e684684586cbf80547ad0d34dae759f66881c802173ac67361b250817b1a785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T06:22:31.510 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T06:22:31.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:31.550 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T06:22:31.550 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-16T06:22:31.550 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-16T06:22:31.831 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:31 smithi190 systemd[1]: Stopping Ceph rgw.foo.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:31.831 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:31 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:22:31.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi190.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.190:8443: connect: connection refused" 2024-09-16T06:22:31.832 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:31 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:22:31.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-16T06:22:32.096 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:31 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a[77603]: 2024-09-16T06:22:31.830+0000 7f4d93ce5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T06:22:32.097 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:31 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a[77603]: 2024-09-16T06:22:31.830+0000 7f4d97550880 -1 shutting down 2024-09-16T06:22:42.097 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:41 smithi190 bash[90730]: time="2024-09-16T06:22:41Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-16T06:22:42.097 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:41 smithi190 podman[90730]: 2024-09-16 06:22:41.853975121 +0000 UTC m=+10.110104010 container died 9bb2fcabd973e1b6c89bbff9531f854ce0b88d87b348b4c5bd91dd598b3c1b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T06:22:42.097 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:41 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:22:41.625Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi190.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.190:8443: connect: connection refused" 2024-09-16T06:22:42.098 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:41 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:22:41.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi190.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.190:8443: connect: connection refused" 2024-09-16T06:22:42.098 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:41 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:22:41.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-16T06:22:42.596 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:42 smithi190 podman[90730]: 2024-09-16 06:22:42.419375187 +0000 UTC m=+10.675504109 container cleanup 9bb2fcabd973e1b6c89bbff9531f854ce0b88d87b348b4c5bd91dd598b3c1b18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD) 2024-09-16T06:22:42.597 INFO:journalctl@ceph.rgw.foo.a.smithi190.stdout:Sep 16 06:22:42 smithi190 bash[90730]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-16T06:22:42.832 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-16T06:22:42.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:42.869 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-16T06:22:42.869 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-16T06:22:42.869 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a 2024-09-16T06:22:43.164 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:42 smithi196 systemd[1]: Stopping Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.163Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T06:22:43.429 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T06:22:43.430 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T06:22:43.430 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T06:22:43.430 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a[73077]: ts=2024-09-16T06:22:43.164Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T06:22:43.430 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 podman[75031]: 2024-09-16 06:22:43.174265835 +0000 UTC m=+0.110445566 container died 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:43.765 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 podman[75031]: 2024-09-16 06:22:43.602817636 +0000 UTC m=+0.538997409 container cleanup 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:43.765 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 bash[75031]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T06:22:44.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-16T06:22:44.046 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:43 smithi196 podman[75048]: 2024-09-16 06:22:43.795533205 +0000 UTC m=+0.623093770 container remove 0db178c80a94092d7e0cd6862effbc41582511f38c782554ff4128d52bca684e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:44.046 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:44 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T06:22:44.046 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:44 smithi196 systemd[1]: Stopped Ceph prometheus.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:22:44.046 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 16 06:22:44 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.170s CPU time. 2024-09-16T06:22:44.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T06:22:44.058 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-16T06:22:44.058 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd41205c-73f2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T06:22:44.202 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:22:46.035 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:45 smithi190 systemd[1]: Stopping Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:46.343 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a[88238]: ts=2024-09-16T06:22:46.035Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T06:22:46.343 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 podman[91203]: 2024-09-16 06:22:46.056077817 +0000 UTC m=+0.115272720 container died dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:46.597 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 podman[91203]: 2024-09-16 06:22:46.433296128 +0000 UTC m=+0.492491035 container cleanup dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:46.597 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 bash[91203]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T06:22:47.089 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 podman[91220]: 2024-09-16 06:22:46.676999938 +0000 UTC m=+0.618640398 container remove dab042f9b7c2b61bc50d3e77d9863f0027e74667b24ad29cc771a90a9a64ec4d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:47.089 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 podman[91220]: 2024-09-16 06:22:46.769201802 +0000 UTC m=+0.710842262 volume remove 17d006d4de38c084fcb3a7f4699d96821edf1049e416d8db45dfacf4b97b25ab 2024-09-16T06:22:47.089 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T06:22:47.089 INFO:journalctl@ceph.alertmanager.a.smithi190.stdout:Sep 16 06:22:46 smithi190 systemd[1]: Stopped Ceph alertmanager.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:22:47.347 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:47 smithi190 systemd[1]: Stopping Ceph node-exporter.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:47.697 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:47 smithi190 podman[91334]: 2024-09-16 06:22:47.418528314 +0000 UTC m=+0.190565733 container died 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:48.048 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:47 smithi190 podman[91334]: 2024-09-16 06:22:47.797124788 +0000 UTC m=+0.569162207 container cleanup 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:48.048 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:47 smithi190 bash[91334]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a 2024-09-16T06:22:48.347 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:48 smithi190 podman[91351]: 2024-09-16 06:22:48.048490205 +0000 UTC m=+0.628698281 container remove 1e2d784049e5c33800d337d27b9e170fb2a19249584dc6812f43f0dcf753369c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T06:22:48.347 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:48 smithi190 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T06:22:48.348 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:48 smithi190 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-16T06:22:48.348 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:48 smithi190 systemd[1]: Stopped Ceph node-exporter.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:22:48.348 INFO:journalctl@ceph.node-exporter.a.smithi190.stdout:Sep 16 06:22:48 smithi190 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 1.921s CPU time. 2024-09-16T06:22:48.870 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd41205c-73f2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T06:22:49.023 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:22:50.484 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:22:50 smithi196 systemd[1]: Stopping Ceph iscsi.iscsi.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:22:50.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:22:50 smithi196 ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a[67787]: debug Shutdown received 2024-09-16T06:23:00.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:00 smithi196 bash[75415]: time="2024-09-16T06:23:00Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-16T06:23:00.751 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:00 smithi196 podman[75415]: 2024-09-16 06:23:00.504466529 +0000 UTC m=+10.139907216 container died cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T06:23:01.015 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:00 smithi196 podman[75415]: 2024-09-16 06:23:00.955018247 +0000 UTC m=+10.590458940 container cleanup cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T06:23:01.015 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:00 smithi196 bash[75415]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-16T06:23:01.515 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 podman[75437]: 2024-09-16 06:23:01.248184989 +0000 UTC m=+0.743716837 container remove cc18aab471497c21d373902220be401c0aa3c2ec2f25a810477bb96e7b893f14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T06:23:01.515 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-16T06:23:01.825 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 podman[75495]: 2024-09-16 06:23:01.539569451 +0000 UTC m=+0.122684105 container died cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T06:23:02.185 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 podman[75495]: 2024-09-16 06:23:01.933562929 +0000 UTC m=+0.516677582 container cleanup cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T06:23:02.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 bash[75495]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-16T06:23:02.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-16T06:23:02.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 67616 (conmon) remains running after unit stopped. 2024-09-16T06:23:02.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 75515 (podman) remains running after unit stopped. 2024-09-16T06:23:02.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 systemd[1]: Stopped Ceph iscsi.iscsi.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:23:02.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:01 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.312s CPU time. 2024-09-16T06:23:02.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:02 smithi196 systemd[1]: Stopping Ceph grafana.a for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:23:02.515 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 16 06:23:02 smithi196 podman[75515]: 2024-09-16 06:23:02.188882312 +0000 UTC m=+0.651459647 container remove cffe4f71e94cddcef21cd1c94e638f499fc7eeab4f32207b5f8584a4b273234a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T06:23:02.879 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:02 smithi196 podman[75640]: 2024-09-16 06:23:02.625989879 +0000 UTC m=+0.119106279 container died 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a, version=8.5, name=ubi8, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, distribution-scope=public, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-16T06:23:03.155 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:02 smithi196 podman[75640]: 2024-09-16 06:23:02.94588147 +0000 UTC m=+0.438997866 container cleanup 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., architecture=x86_64, description=Ceph Grafana Container, io.openshift.tags=base rhel8, io.openshift.expose-services=) 2024-09-16T06:23:03.155 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:02 smithi196 bash[75640]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a 2024-09-16T06:23:03.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:03 smithi196 podman[75658]: 2024-09-16 06:23:03.155301158 +0000 UTC m=+0.531343203 container remove 09172c9fb65a01d2c3af9b599e7b2de272c8f6062ad6ed27211999fc24766abd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, distribution-scope=public, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, vendor=Red Hat, Inc., version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, release=239.1651231664, vcs-type=git) 2024-09-16T06:23:03.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:03 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T06:23:03.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:03 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-16T06:23:03.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:03 smithi196 systemd[1]: Stopped Ceph grafana.a for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:23:03.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 16 06:23:03 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@grafana.a.service: Consumed 3.689s CPU time. 2024-09-16T06:23:03.820 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:03 smithi196 systemd[1]: Stopping Ceph node-exporter.b for bd41205c-73f2-11ef-bceb-c7b262605968... 2024-09-16T06:23:04.083 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:03 smithi196 podman[75768]: 2024-09-16 06:23:03.82113981 +0000 UTC m=+0.119481250 container died 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T06:23:04.515 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 podman[75768]: 2024-09-16 06:23:04.26634207 +0000 UTC m=+0.564683500 container cleanup 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T06:23:04.515 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 bash[75768]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b 2024-09-16T06:23:04.841 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 podman[75786]: 2024-09-16 06:23:04.542319351 +0000 UTC m=+0.725076578 container remove 7c14417c9e26caeb1b02a9d771ec065e88c7dd8bc3ea4d5d4fa16cec07ff6c66 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bd41205c-73f2-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T06:23:04.841 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-16T06:23:04.841 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-16T06:23:04.841 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 systemd[1]: Stopped Ceph node-exporter.b for bd41205c-73f2-11ef-bceb-c7b262605968. 2024-09-16T06:23:04.841 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 16 06:23:04 smithi196 systemd[1]: ceph-bd41205c-73f2-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 1.888s CPU time. 2024-09-16T06:23:05.342 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:23:05.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T06:23:05.403 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T06:23:05.405 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536/remote/smithi190/crash 2024-09-16T06:23:05.406 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/crash -- . 2024-09-16T06:23:05.451 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T06:23:05.451 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-09-16T06:23:05.453 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536/remote/smithi196/crash 2024-09-16T06:23:05.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/crash -- . 2024-09-16T06:23:05.493 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T06:23:05.493 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-16T06:23:05.494 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T06:23:05.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v CEPHADM_AGENT_DOWN | head -n 1 2024-09-16T06:23:05.525 INFO:teuthology.orchestra.run.smithi190.stderr:grep: /var/log/ceph/bd41205c-73f2-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-16T06:23:05.527 INFO:tasks.cephadm:Compressing logs... 2024-09-16T06:23:05.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:23:05.570 DEBUG:teuthology.orchestra.run.smithi196:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:23:05.594 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T06:23:05.595 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T06:23:07.063 INFO:tasks.cephadm:Archiving logs... 2024-09-16T06:23:07.063 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536/remote/smithi190/log 2024-09-16T06:23:07.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T06:23:07.339 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536/remote/smithi196/log 2024-09-16T06:23:07.340 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T06:23:07.496 INFO:tasks.cephadm:Removing cluster... 2024-09-16T06:23:07.496 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd41205c-73f2-11ef-bceb-c7b262605968 --force 2024-09-16T06:23:07.636 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:23:07.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd41205c-73f2-11ef-bceb-c7b262605968 --force 2024-09-16T06:23:07.916 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: bd41205c-73f2-11ef-bceb-c7b262605968 2024-09-16T06:23:08.044 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T06:23:08.044 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T06:23:08.061 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T06:23:08.077 INFO:tasks.cephadm:Teardown complete 2024-09-16T06:23:08.077 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-16T06:23:08.087 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_1... 2024-09-16T06:23:08.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_1 2024-09-16T06:23:08.261 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-16T06:23:08.263 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_2... 2024-09-16T06:23:08.263 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_2 2024-09-16T06:23:08.414 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-16T06:23:08.415 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_3... 2024-09-16T06:23:08.416 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_3 2024-09-16T06:23:08.571 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-16T06:23:08.572 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi190:/dev/vg_nvme/lv_4... 2024-09-16T06:23:08.573 DEBUG:teuthology.orchestra.run.smithi190:> sudo nvme disconnect -n lv_4 2024-09-16T06:23:08.733 INFO:teuthology.orchestra.run.smithi190.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-16T06:23:08.735 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-16T06:23:08.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/scratch_devs 2024-09-16T06:23:08.759 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_1... 2024-09-16T06:23:08.760 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_1 2024-09-16T06:23:08.917 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-16T06:23:08.919 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_2... 2024-09-16T06:23:08.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_2 2024-09-16T06:23:09.077 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-16T06:23:09.079 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_3... 2024-09-16T06:23:09.079 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_3 2024-09-16T06:23:09.265 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-16T06:23:09.270 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_4... 2024-09-16T06:23:09.270 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_4 2024-09-16T06:23:09.409 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-16T06:23:09.411 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-16T06:23:09.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/scratch_devs 2024-09-16T06:23:09.440 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T06:23:09.449 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T06:23:09.449 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:23:09.453 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:23:09.465 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-16T06:23:09.474 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T06:23:09.474 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-16T06:23:09.474 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1241us[+1245us] +/- 53ms 2024-09-16T06:23:09.474 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1513us[-1510us] +/- 71ms 2024-09-16T06:23:09.474 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -244us[ -241us] +/- 74ms 2024-09-16T06:23:09.474 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:23:09.497 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-16T06:23:09.503 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T06:23:09.503 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-16T06:23:09.503 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1262us[+1288us] +/- 53ms 2024-09-16T06:23:09.503 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -1545us[-1519us] +/- 71ms 2024-09-16T06:23:09.503 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -269us[ -269us] +/- 74ms 2024-09-16T06:23:09.503 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T06:23:09.504 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T06:23:09.513 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T06:23:09.513 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T06:23:09.523 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T06:23:09.600 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T06:23:09.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T06:23:09.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:23:09.707 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T06:23:09.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:23:09.737 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T06:23:09.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T06:23:09.753 INFO:teuthology.task.internal:Duration was 839.788813 seconds 2024-09-16T06:23:09.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T06:23:09.762 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T06:23:09.763 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T06:23:09.765 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T06:23:09.802 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T06:23:09.817 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T06:23:10.058 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T06:23:10.059 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-09-16T06:23:10.059 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T06:23:10.079 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-16T06:23:10.079 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T06:23:10.124 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T06:23:10.125 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:23:10.127 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:23:10.191 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T06:23:10.191 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T06:23:10.194 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T06:23:10.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T06:23:10.468 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T06:23:10.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T06:23:10.493 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T06:23:10.518 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T06:23:10.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:23:10.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:23:10.556 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-09-16T06:23:10.582 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-16T06:23:10.629 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:23:10.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:23:10.643 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:23:10.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:23:10.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T06:23:10.693 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T06:23:10.693 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536/remote/smithi190 2024-09-16T06:23:10.694 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T06:23:10.733 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906536/remote/smithi196 2024-09-16T06:23:10.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T06:23:10.770 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T06:23:10.771 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T06:23:10.773 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T06:23:10.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T06:23:10.833 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T06:23:10.833 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T06:23:10.841 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T06:23:10.842 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T06:23:10.844 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T06:23:10.857 INFO:teuthology.orchestra.run.smithi190.stdout: 265613 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 06:23 /home/ubuntu/cephtest 2024-09-16T06:23:10.877 INFO:teuthology.orchestra.run.smithi196.stdout: 265815 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 06:23 /home/ubuntu/cephtest 2024-09-16T06:23:10.878 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T06:23:10.888 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T06:23:10.945 INFO:teuthology.run:Summary data: description: rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 839.7888126373291 owner: scheduled_teuthology@teuthology success: true 2024-09-16T06:23:10.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T06:23:11.033 INFO:teuthology.run:pass